Skip to content

GitLab

D
docker-image
Commit f47954a6 authored by caiyanming's avatar caiyanming
Browse files
Options

add mongo 

Change-Id: Ie156da396271cb9ace1dbdf4873adf2ff177b784
parent 4234d57c
Expand all Show whitespace changes
Inline Side-by-side
Showing with 477 additions and 515 deletions
hbase/Dockerfile deleted 100644 → 0
View file @ 4234d57c
# HBase in Docker
#
# Version 0.4
# http://docs.docker.io/en/latest/use/builder/
FROM ubuntu:bionic
MAINTAINER Dave Beckett <dave@dajobe.org>
COPY sources.list /etc/apt/sources.list
COPY *.sh /build/
ENV HBASE_VERSION 2.2.4
ADD hbase-$HBASE_VERSION-bin.tar.gz /opt/
RUN /build/prepare-hbase.sh && \
cd /opt/hbase && /build/build-hbase.sh \
cd / && /build/cleanup-hbase.sh && rm -rf /build
VOLUME /data
ADD ./hbase-site.xml /opt/hbase/conf/hbase-site.xml
ADD ./zoo.cfg /opt/hbase/conf/zoo.cfg
ADD ./replace-hostname /opt/replace-hostname
ADD ./hbase-server /opt/hbase-server
# REST API
EXPOSE 8080
# REST Web UI at :8085/rest.jsp
EXPOSE 8085
# Thrift API
EXPOSE 9090
# Thrift Web UI at :9095/thrift.jsp
EXPOSE 9095
# HBase's Embedded zookeeper cluster
EXPOSE 2181
# HBase Master web UI at :16010/master-status; ZK at :16010/zk.jsp
EXPOSE 16010
CMD ["/opt/hbase-server"]
hbase/Makefile deleted 100644 → 0
View file @ 4234d57c
IMAGE_NAME=dajobe/hbase
IMAGE_TAG=latest
HBASE_VERSION=$(shell awk '/^ENV HBASE_VERSION/ {print $3}' Dockerfile)
build:
@echo "Building hbase docker image $(HBASE_VERSION)"
docker build -t $(IMAGE_NAME) .
# This won't work unless you have already set up the repository config
push:
@echo "Pushing image to https://hub.docker.com/"
docker push $(IMAGE_NAME):$(IMAGE_TAG)
hbase/README.md deleted 100644 → 0
View file @ 4234d57c
HBase in Docker
===============
This configuration builds a docker container to run HBase (with
embedded Zookeeper) running on the files inside the container.
NOTE
----
The approach here requires editing the local server's `/etc/hosts`
file to add an entry for the container hostname. This is because
HBase uses hostnames to pass connection data back out of the
container (from it's internal Zookeeper).
Hopefully this can be improved with Docker's newer networking
but this hasn't been fixed yet.
Build Image
-----------
$ docker build -t dajobe/hbase .
Pull image
----------
If you want to pull the image already built then use this
$ docker pull dajobe/hbase
More details at https://hub.docker.com/r/dajobe/hbase/
Run HBase
---------
To run HBase by hand:
$ mkdir data
$ id=$(docker run --name=hbase-docker -h hbase-docker -d -v $PWD/data:/data dajobe/hbase)
To run it and adjust the host system's locally by editing
`/etc/hosts` to alias the DNS hostname 'hbase-docker' to the
container, use this:
$ ./start-hbase.sh
This will require you to enter your sudo password to edit the host
machine's `/etc/hosts` file
If you want to run multiple hbase dockers on the same host, you can
give them different hostnames with the '-h' / '--hostname' argument.
You may have to give them different ports though. Not tested.
If you want to customize the hostname used, set the
`HBASE_DOCKER_HOSTNAME` envariable on the docker command line
Find Hbase status
-----------------
Master status if docker container DNS name is 'hbase-docker'
http://hbase-docker:16010/master-status
The region servers status pages are linked from the above page.
Thrift UI
http://hbase-docker:9095/thrift.jsp
REST server UI
http://hbase-docker:8085/rest.jsp
(Embedded) Zookeeper status
http://hbase-docker:16010/zk.jsp
See HBase Logs
--------------
If you want to see the latest logs live use:
$ docker attach $id
Then ^C to detach.
To see all the logs since the HBase server started, use:
$ docker logs $id
and ^C to detach again.
To see the individual log files without using `docker`, look into
the data volume dir eg $PWD/data/logs if invoked as above.
Test HBase is working via python over Thrift
--------------------------------------------
Here I am connecting to a docker container with the name 'hbase-docker'
(such as created by the start-hbase.sh script). The port 9090 is the
Thrift API port because [Happybase][1] [2] uses Thrift to talk to HBase.
$ python
Python 2.7.15 (default, Jan 12 2019, 21:07:57)
[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import happybase
>>> connection = happybase.Connection('hbase-docker', 9090)
>>> connection.create_table('table-name', { 'family': dict() } )
>>> connection.tables()
['table-name']
>>> table = connection.table('table-name')
>>> table.put('row-key', {'family:qual1': 'value1', 'family:qual2': 'value2'})
>>> for k, data in table.scan():
... print k, data
...
row-key {'family:qual1': 'value1', 'family:qual2': 'value2'}
>>>
(Simple install for happybase: `sudo pip install happybase` although I
use `pip install --user happybase` to get it just for me)
Test HBase is working from Java
-------------------------------
$ docker run --rm -it --link $id:hbase-docker dajobe/hbase hbase shell
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.1.2, r1dfc418f77801fbfb59a125756891b9100c1fc6d, Sun Dec 30 21:45:09 PST 2018
Took 0.0472 seconds
hbase(main):001:0> status
1 active master, 0 backup masters, 1 servers, 0 dead, 2.0000 average load
Took 0.7255 seconds
hbase(main):002:0> list
TABLE
table-name
1 row(s)
Took 0.0509 seconds
=> ["table-name"]
hbase(main):003:0>
Showing the `table-name` table made in the happybase example above.
Alternatively if you have the Hbase distribution available on the
host you can use `bin/hbase shell` if the hbase configuration has
been set up to connect to host `hbase-docker` zookeeper port 2181 to
get the servers via configuration property `hbase.zookeeper.quorum`
Proxy HBase UIs locally
-----------------------
If you are running docker on a remote machine, it is handy to see
these server-private urls in a local browser so here is a
~/.ssh/config fragment to do that
Host my-docker-server
Hostname 1.2.3.4
LocalForward 127.0.0.1:16010 127.0.0.1:16010
LocalForward 127.0.0.1:9095 127.0.0.1:9095
LocalForward 127.0.0.1:8085 127.0.0.1:8085
When you `ssh my-docker-server` ssh connects to the docker server and
forwards request on your local machine on ports 16010 / 16030 to the
remote ports that are attached to the hbase container.
The bottom line, you can use these URLs to see what's going on:
* http://localhost:16010/master-status for the Master Server
* http://localhost:9095/thrift.jsp for the thrift UI
* http://localhost:8085/rest.jsp for the REST server UI
* http://localhost:16010/zk.jsp for the embedded Zookeeper
to see what's going on in the container and since both your local
machine and the container are using localhost (aka 127.0.0.1), even
the links work!
Notes
-----
[1] http://happybase.readthedocs.org/en/latest/
[2] https://github.com/wbolster/happybase
hbase/build-hbase.sh deleted 100755 → 0
View file @ 4234d57c
#!/bin/sh -xe
. /build/config-hbase.sh
here=$(pwd)
# delete files that are not needed to run hbase
rm -rf docs *.txt LEGAL
rm -f */*.cmd
# Set Java home for hbase servers
sed -i "s,^. export JAVA_HOME.*,export JAVA_HOME=$JAVA_HOME," conf/hbase-env.sh
# Set interactive shell defaults
cat > /etc/profile.d/defaults.sh <<EOF
JAVA_HOME=$JAVA_HOME
export JAVA_HOME
EOF
cd /usr/bin
ln -sf $here/bin/* .
hbase/cleanup-hbase.sh deleted 100755 → 0
View file @ 4234d57c
#!/bin/sh -xe
. /build/config-hbase.sh
AUTO_ADDED_PACKAGES=`apt-mark showauto`
apt-get remove --purge -y $HBASE_BUILD_PACKAGES $AUTO_ADDED_PACKAGES
# Install the run-time dependencies
apt-get install $minimal_apt_get_args $HBASE_RUN_PACKAGES
# . /build/cleanup.sh
rm -rf /tmp/* /var/tmp/*
apt-get clean
rm -rf /var/lib/apt/lists/*
hbase/config-hbase.sh deleted 100755 → 0
View file @ 4234d57c
# This file intended to be sourced
# . /build/config.sh
# This is the definitive site and incredibly slow
HBASE_DIST="http://archive.apache.org/dist/hbase"
# This is a mirror site and faster but every new release breaks all
# existing links.
# HBASE_DIST="https://www-us.apache.org/dist/hbase"
HBASE_DIST="https://mirrors.tuna.tsinghua.edu.cn/apache/hbase"
# Prevent initramfs updates from trying to run grub and lilo.
export INITRD=no
export DEBIAN_FRONTEND=noninteractive
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
minimal_apt_get_args='-y --no-install-recommends'
## Build time dependencies ##
HBASE_BUILD_PACKAGES="curl"
# Core list from docs
#HBASE_BUILD_PACKAGES="$HBASE_BUILD_PACKAGES "
# Optional:
#HBASE_BUILD_PACKAGES="$HBASE_BUILD_PACKAGES "
## Run time dependencies ##
HBASE_RUN_PACKAGES="openjdk-8-jre-headless"
hbase/hbase-server deleted 100755 → 0
View file @ 4234d57c
#!/bin/bash
# Start HBase
# 1) thrift API/Web UI
# 2) hbase master (which runs the local region server)
logs_dir=/data/logs
# Prepare environment
/opt/replace-hostname
mkdir -p $logs_dir /data/hbase /data/run
# Thrift server (background)
# Ports: 9090 API and 9095 UI
echo "hbase thrift start logging to $logs_dir/hbase-thrift.log"
hbase thrift start > $logs_dir/hbase-thrift.log 2>&1 &
# REST server (background)
# Ports: 8080 API
echo "hbase rest start logging to $logs_dir/hbase-rest.log"
hbase rest start > $logs_dir/hbase-rest.log 2>&1 &
# Master server (Foreground) that also starts the region server
# Ports: Master: 16000 API, 16010 UI; 2181 ZK; Region: 16020 API, 16030 UI
echo "hbase master start logging to $logs_dir/hbase-master.log"
exec hbase master start 2>&1 | tee $logs_dir/hbase-master.log
hbase/hbase-site.xml deleted 100644 → 0
View file @ 4234d57c
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>hbase.zookeeper.quorum</name>
<value>hbase-docker</value>
</property>
<property>
<name>hbase.rootdir</name>
<value>file:////data/hbase</value>
</property>
<property>
<name>hbase.master.info.bindAddress</name>
<value>hbase-docker</value>
</property>
<property>
<name>hbase.regionserver.info.bindAddress</name>
<value>hbase-docker</value>
</property>
</configuration>
hbase/prepare-hbase.sh deleted 100755 → 0
View file @ 4234d57c
#!/bin/sh -xe
. /build/config-hbase.sh
apt-get update -y
apt-get install $minimal_apt_get_args $HBASE_BUILD_PACKAGES
cd /opt
#curl -SL $HBASE_DIST/$HBASE_VERSION/hbase-$HBASE_VERSION-bin.tar.gz | tar -x -z && mv hbase-${HBASE_VERSION} hbase
mv hbase-${HBASE_VERSION} hbase
hbase/replace-hostname deleted 100755 → 0
View file @ 4234d57c
#!/bin/bash
#
# Script that replaces the default hostname in files with the environments
# ${HOSTNAME} variable.
#
# This script is intended to be run before starting hbase-server to ensure
# that the hostname matches the configured environment variable. i.e.
# the -h --hostname flag.
#
declare -a files=(
'/opt/hbase/conf/hbase-site.xml'
'/opt/hbase/conf/zoo.cfg'
)
# Optional custom hostname replacement
REPLACEMENT_HOSTNAME=${HBASE_DOCKER_HOSTNAME:-$HOSTNAME}
for file in "${files[@]}"; do
if [ -f "${file}.bak" ]; then
cp "${file}.bak" "${file}"
else
cp "${file}" "${file}.bak"
fi
sed -i "s/hbase-docker/${REPLACEMENT_HOSTNAME}/g" "${file}"
done
hbase/sources.list deleted 100644 → 0
View file @ 4234d57c
# deb cdrom:[Ubuntu 16.04 LTS _Xenial Xerus_ - Release amd64 (20160420.1)]/ bionic main restricted
deb-src http://archive.ubuntu.com/ubuntu bionic main restricted #Added by software-properties
deb http://mirrors.aliyun.com/ubuntu/ bionic main restricted
deb-src http://mirrors.aliyun.com/ubuntu/ bionic main restricted multiverse universe #Added by software-properties
deb http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted multiverse universe #Added by software-properties
deb http://mirrors.aliyun.com/ubuntu/ bionic universe
deb http://mirrors.aliyun.com/ubuntu/ bionic-updates universe
deb http://mirrors.aliyun.com/ubuntu/ bionic multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-updates multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse #Added by software-properties
deb http://archive.canonical.com/ubuntu bionic partner
deb-src http://archive.canonical.com/ubuntu bionic partner
deb http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted multiverse universe #Added by software-properties
deb http://mirrors.aliyun.com/ubuntu/ bionic-security universe
deb http://mirrors.aliyun.com/ubuntu/ bionic-security multiverse
hbase/start-hbase.sh deleted 100755 → 0
View file @ 4234d57c
#!/bin/bash -e
#
# Script to start docker and update the /etc/hosts file to point to
# the hbase-docker container
#
# hbase thrift and master server logs are written to the local
# logs directory
#
echo "Starting HBase container"
data_dir=$PWD/data
rm -rf $data_dir
mkdir -p $data_dir
id=$(docker run --name=hbase-docker -h hbase-docker -d -v $data_dir:/data dajobe/hbase)
echo "Container has ID $id"
# Get the hostname and IP inside the container
docker inspect $id > config.json
docker_hostname=$(python -c 'from __future__ import print_function; import json; c=json.load(open("config.json")); print(c[0]["Config"]["Hostname"])')
docker_ip=$(python -c 'from __future__ import print_function; import json; c=json.load(open("config.json")); print(c[0]["NetworkSettings"]["IPAddress"])')
rm -f config.json
echo "Updating /etc/hosts to make hbase-docker point to $docker_ip ($docker_hostname)"
if grep 'hbase-docker' /etc/hosts >/dev/null; then
sudo sed -i.bak "s/^.*hbase-docker.*\$/$docker_ip hbase-docker $docker_hostname/" /etc/hosts
else
sudo sh -c "echo '\n$docker_ip hbase-docker $docker_hostname' >> /etc/hosts"
fi
echo "Now connect to hbase at localhost on the standard ports"
echo " ZK 2181, Thrift 9090, Master 16000, Region 16020"
echo "Or connect to host hbase-docker (in the container) on the same ports"
echo ""
echo "For docker status:"
echo "$ id=$id"
echo "$ docker inspect \$id"
hbase/test_hbase.py deleted 100644 → 0
View file @ 4234d57c
# https://happybase.readthedocs.org/en/latest/
# https://github.com/wbolster/happybase
import happybase
def main():
HOST='hbase-docker'
PORT=9090
# Will create and then delete this table
TABLE_NAME='table-name'
ROW_KEY='row-key'
connection = happybase.Connection(HOST, PORT)
tables = connection.tables()
print "HBase has tables {0}".format(tables)
if TABLE_NAME not in tables:
print "Creating table {0}".format(TABLE_NAME)
connection.create_table(TABLE_NAME, { 'family': dict() } )
table = connection.table(TABLE_NAME)
print "Storing values with row key '{0}'".format(ROW_KEY)
table.put(ROW_KEY, {'family:qual1': 'value1',
'family:qual2': 'value2'})
print "Getting values for row key '{0}'".format(ROW_KEY)
row = table.row(ROW_KEY)
print row['family:qual1']
print "Printing rows with keys '{0}' and row-key-2".format(ROW_KEY)
for key, data in table.rows([ROW_KEY, 'row-key-2']):
print key, data
print "Scanning rows with prefix 'row'"
for key, data in table.scan(row_prefix='row'):
print key, data # prints 'value1' and 'value2'
print "Deleting row '{0}'".format(ROW_KEY)
row = table.delete(ROW_KEY)
print "Deleting table {0}".format(TABLE_NAME)
connection.delete_table(TABLE_NAME, disable=True)
if __name__ == "__main__":
main()
hbase/zoo.cfg deleted 100644 → 0
View file @ 4234d57c
clientPort=2181
clientPortAddress=hbase-docker
server.1=hbase-docker:2181
mongo/Dockerfile 0 → 100644
View file @ f47954a6
FROM ubuntu:18.04
MAINTAINER xt
COPY sources.list /etc/apt/sources.list
# add our user and group first to make sure their IDs get assigned consistently, regardless of whatever dependencies get added
RUN groupadd -r mongodb && useradd -r -g mongodb mongodb
RUN set -eux; \
apt-get update; \
apt-get install -y --no-install-recommends \
ca-certificates \
jq \
numactl \
; \
if ! command -v ps > /dev/null; then \
apt-get install -y --no-install-recommends procps; \
fi; \
rm -rf /var/lib/apt/lists/*
# grab gosu for easy step-down from root (https://github.com/tianon/gosu/releases)
ENV GOSU_VERSION 1.12
# grab "js-yaml" for parsing mongod's YAML config files (https://github.com/nodeca/js-yaml/releases)
ENV JSYAML_VERSION 3.13.1
RUN set -ex; \
\
savedAptMark="$(apt-mark showmanual)"; \
apt-get update; \
apt-get install -y --no-install-recommends \
wget \
; \
if ! command -v gpg > /dev/null; then \
apt-get install -y --no-install-recommends gnupg dirmngr; \
savedAptMark="$savedAptMark gnupg dirmngr"; \
elif gpg --version | grep -q '^gpg (GnuPG) 1\.'; then \
# "This package provides support for HKPS keyservers." (GnuPG 1.x only)
apt-get install -y --no-install-recommends gnupg-curl; \
fi; \
rm -rf /var/lib/apt/lists/*; \
\
dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')"; \
wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch"; \
wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch.asc"; \
export GNUPGHOME="$(mktemp -d)"; \
gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4; \
gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; \
command -v gpgconf && gpgconf --kill all || :; \
rm -r "$GNUPGHOME" /usr/local/bin/gosu.asc; \
# \
# wget -O /js-yaml.js "https://github.com/nodeca/js-yaml/raw/${JSYAML_VERSION}/dist/js-yaml.js"; \
# TODO some sort of download verification here
\
apt-mark auto '.*' > /dev/null; \
apt-mark manual $savedAptMark > /dev/null; \
apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false; \
\
# smoke test
chmod +x /usr/local/bin/gosu; \
gosu --version; \
gosu nobody true
COPY js-yaml.min.js /js-yaml.js
RUN mkdir /docker-entrypoint-initdb.d
ENV GPG_KEYS E162F504A20CDF15827F718D4B7C549A058F8B6B
RUN set -ex; \
export GNUPGHOME="$(mktemp -d)"; \
for key in $GPG_KEYS; do \
gpg --batch --keyserver ha.pool.sks-keyservers.net --recv-keys "$key"; \
done; \
gpg --batch --export $GPG_KEYS > /etc/apt/trusted.gpg.d/mongodb.gpg; \
command -v gpgconf && gpgconf --kill all || :; \
rm -r "$GNUPGHOME"; \
apt-key list
# Allow build-time overrides (eg. to build image with MongoDB Enterprise version)
# Options for MONGO_PACKAGE: mongodb-org OR mongodb-enterprise
# Options for MONGO_REPO: repo.mongodb.org OR repo.mongodb.com
# Example: docker build --build-arg MONGO_PACKAGE=mongodb-enterprise --build-arg MONGO_REPO=repo.mongodb.com .
ARG MONGO_PACKAGE=mongodb-org
ARG MONGO_REPO=repo.mongodb.org
ENV MONGO_PACKAGE=${MONGO_PACKAGE} MONGO_REPO=${MONGO_REPO}
ENV MONGO_MAJOR 4.2
ENV MONGO_VERSION 4.2.6
# bashbrew-architectures:amd64 arm64v8 s390x
RUN echo "deb http://$MONGO_REPO/apt/ubuntu bionic/${MONGO_PACKAGE%-unstable}/$MONGO_MAJOR multiverse" | tee "/etc/apt/sources.list.d/${MONGO_PACKAGE%-unstable}.list"
RUN set -x \
# installing "mongodb-enterprise" pulls in "tzdata" which prompts for input
&& export DEBIAN_FRONTEND=noninteractive \
&& apt-get update \
&& apt-get install -y \
${MONGO_PACKAGE}=$MONGO_VERSION \
${MONGO_PACKAGE}-server=$MONGO_VERSION \
${MONGO_PACKAGE}-shell=$MONGO_VERSION \
${MONGO_PACKAGE}-mongos=$MONGO_VERSION \
${MONGO_PACKAGE}-tools=$MONGO_VERSION \
&& rm -rf /var/lib/apt/lists/* \
&& rm -rf /var/lib/mongodb \
&& mv /etc/mongod.conf /etc/mongod.conf.orig
RUN mkdir -p /data/db /data/configdb \
&& chown -R mongodb:mongodb /data/db /data/configdb
VOLUME /data/db /data/configdb
COPY docker-entrypoint.sh /usr/local/bin/
ENTRYPOINT ["docker-entrypoint.sh"]
EXPOSE 27017
CMD ["mongod"]
mongo/docker-entrypoint.sh 0 → 100755
View file @ f47954a6
#!/bin/bash
set -Eeuo pipefail
if [ "${1:0:1}" = '-' ]; then
set -- mongod "$@"
fi
originalArgOne="$1"
# allow the container to be started with `--user`
# all mongo* commands should be dropped to the correct user
if [[ "$originalArgOne" == mongo* ]] && [ "$(id -u)" = '0' ]; then
if [ "$originalArgOne" = 'mongod' ]; then
find /data/configdb /data/db \! -user mongodb -exec chown mongodb '{}' +
fi
# make sure we can write to stdout and stderr as "mongodb"
# (for our "initdb" code later; see "--logpath" below)
chown --dereference mongodb "/proc/$$/fd/1" "/proc/$$/fd/2" || :
# ignore errors thanks to https://github.com/docker-library/mongo/issues/149
exec gosu mongodb "$BASH_SOURCE" "$@"
fi
# you should use numactl to start your mongod instances, including the config servers, mongos instances, and any clients.
# https://docs.mongodb.com/manual/administration/production-notes/#configuring-numa-on-linux
if [[ "$originalArgOne" == mongo* ]]; then
numa='numactl --interleave=all'
if $numa true &> /dev/null; then
set -- $numa "$@"
fi
fi
# usage: file_env VAR [DEFAULT]
# ie: file_env 'XYZ_DB_PASSWORD' 'example'
# (will allow for "$XYZ_DB_PASSWORD_FILE" to fill in the value of
# "$XYZ_DB_PASSWORD" from a file, especially for Docker's secrets feature)
file_env() {
local var="$1"
local fileVar="${var}_FILE"
local def="${2:-}"
if [ "${!var:-}" ] && [ "${!fileVar:-}" ]; then
echo >&2 "error: both $var and $fileVar are set (but are exclusive)"
exit 1
fi
local val="$def"
if [ "${!var:-}" ]; then
val="${!var}"
elif [ "${!fileVar:-}" ]; then
val="$(< "${!fileVar}")"
fi
export "$var"="$val"
unset "$fileVar"
}
# see https://github.com/docker-library/mongo/issues/147 (mongod is picky about duplicated arguments)
_mongod_hack_have_arg() {
local checkArg="$1"; shift
local arg
for arg; do
case "$arg" in
"$checkArg"|"$checkArg"=*)
return 0
;;
esac
done
return 1
}
# _mongod_hack_get_arg_val '--some-arg' "$@"
_mongod_hack_get_arg_val() {
local checkArg="$1"; shift
while [ "$#" -gt 0 ]; do
local arg="$1"; shift
case "$arg" in
"$checkArg")
echo "$1"
return 0
;;
"$checkArg"=*)
echo "${arg#$checkArg=}"
return 0
;;
esac
done
return 1
}
declare -a mongodHackedArgs
# _mongod_hack_ensure_arg '--some-arg' "$@"
# set -- "${mongodHackedArgs[@]}"
_mongod_hack_ensure_arg() {
local ensureArg="$1"; shift
mongodHackedArgs=( "$@" )
if ! _mongod_hack_have_arg "$ensureArg" "$@"; then
mongodHackedArgs+=( "$ensureArg" )
fi
}
# _mongod_hack_ensure_no_arg '--some-unwanted-arg' "$@"
# set -- "${mongodHackedArgs[@]}"
_mongod_hack_ensure_no_arg() {
local ensureNoArg="$1"; shift
mongodHackedArgs=()
while [ "$#" -gt 0 ]; do
local arg="$1"; shift
if [ "$arg" = "$ensureNoArg" ]; then
continue
fi
mongodHackedArgs+=( "$arg" )
done
}
# _mongod_hack_ensure_no_arg '--some-unwanted-arg' "$@"
# set -- "${mongodHackedArgs[@]}"
_mongod_hack_ensure_no_arg_val() {
local ensureNoArg="$1"; shift
mongodHackedArgs=()
while [ "$#" -gt 0 ]; do
local arg="$1"; shift
case "$arg" in
"$ensureNoArg")
shift # also skip the value
continue
;;
"$ensureNoArg"=*)
# value is already included
continue
;;
esac
mongodHackedArgs+=( "$arg" )
done
}
# _mongod_hack_ensure_arg_val '--some-arg' 'some-val' "$@"
# set -- "${mongodHackedArgs[@]}"
_mongod_hack_ensure_arg_val() {
local ensureArg="$1"; shift
local ensureVal="$1"; shift
_mongod_hack_ensure_no_arg_val "$ensureArg" "$@"
mongodHackedArgs+=( "$ensureArg" "$ensureVal" )
}
# _js_escape 'some "string" value'
_js_escape() {
jq --null-input --arg 'str' "$1" '$str'
}
jsonConfigFile="${TMPDIR:-/tmp}/docker-entrypoint-config.json"
tempConfigFile="${TMPDIR:-/tmp}/docker-entrypoint-temp-config.json"
_parse_config() {
if [ -s "$tempConfigFile" ]; then
return 0
fi
local configPath
if configPath="$(_mongod_hack_get_arg_val --config "$@")"; then
# if --config is specified, parse it into a JSON file so we can remove a few problematic keys (especially SSL-related keys)
# see https://docs.mongodb.com/manual/reference/configuration-options/
mongo --norc --nodb --quiet --eval "load('/js-yaml.js'); printjson(jsyaml.load(cat($(_js_escape "$configPath"))))" > "$jsonConfigFile"
jq 'del(.systemLog, .processManagement, .net, .security)' "$jsonConfigFile" > "$tempConfigFile"
return 0
fi
return 1
}
dbPath=
_dbPath() {
if [ -n "$dbPath" ]; then
echo "$dbPath"
return
fi
if ! dbPath="$(_mongod_hack_get_arg_val --dbpath "$@")"; then
if _parse_config "$@"; then
dbPath="$(jq -r '.storage.dbPath // empty' "$jsonConfigFile")"
fi
fi
if [ -z "$dbPath" ]; then
if _mongod_hack_have_arg --configsvr "$@" || {
_parse_config "$@" \
&& clusterRole="$(jq -r '.sharding.clusterRole // empty' "$jsonConfigFile")" \
&& [ "$clusterRole" = 'configsvr' ]
}; then
# if running as config server, then the default dbpath is /data/configdb
# https://docs.mongodb.com/manual/reference/program/mongod/#cmdoption-mongod-configsvr
dbPath=/data/configdb
fi
fi
: "${dbPath:=/data/db}"
echo "$dbPath"
}
if [ "$originalArgOne" = 'mongod' ]; then
file_env 'MONGO_INITDB_ROOT_USERNAME'
file_env 'MONGO_INITDB_ROOT_PASSWORD'
# pre-check a few factors to see if it's even worth bothering with initdb
shouldPerformInitdb=
if [ "$MONGO_INITDB_ROOT_USERNAME" ] && [ "$MONGO_INITDB_ROOT_PASSWORD" ]; then
# if we have a username/password, let's set "--auth"
_mongod_hack_ensure_arg '--auth' "$@"
set -- "${mongodHackedArgs[@]}"
shouldPerformInitdb='true'
elif [ "$MONGO_INITDB_ROOT_USERNAME" ] || [ "$MONGO_INITDB_ROOT_PASSWORD" ]; then
cat >&2 <<-'EOF'
error: missing 'MONGO_INITDB_ROOT_USERNAME' or 'MONGO_INITDB_ROOT_PASSWORD'
both must be specified for a user to be created
EOF
exit 1
fi
if [ -z "$shouldPerformInitdb" ]; then
# if we've got any /docker-entrypoint-initdb.d/* files to parse later, we should initdb
for f in /docker-entrypoint-initdb.d/*; do
case "$f" in
*.sh|*.js) # this should match the set of files we check for below
shouldPerformInitdb="$f"
break
;;
esac
done
fi
# check for a few known paths (to determine whether we've already initialized and should thus skip our initdb scripts)
if [ -n "$shouldPerformInitdb" ]; then
dbPath="$(_dbPath "$@")"
for path in \
"$dbPath/WiredTiger" \
"$dbPath/journal" \
"$dbPath/local.0" \
"$dbPath/storage.bson" \
; do
if [ -e "$path" ]; then
shouldPerformInitdb=
break
fi
done
fi
if [ -n "$shouldPerformInitdb" ]; then
mongodHackedArgs=( "$@" )
if _parse_config "$@"; then
_mongod_hack_ensure_arg_val --config "$tempConfigFile" "${mongodHackedArgs[@]}"
fi
_mongod_hack_ensure_arg_val --bind_ip 127.0.0.1 "${mongodHackedArgs[@]}"
_mongod_hack_ensure_arg_val --port 27017 "${mongodHackedArgs[@]}"
_mongod_hack_ensure_no_arg --bind_ip_all "${mongodHackedArgs[@]}"
# remove "--auth" and "--replSet" for our initial startup (see https://docs.mongodb.com/manual/tutorial/enable-authentication/#start-mongodb-without-access-control)
# https://github.com/docker-library/mongo/issues/211
_mongod_hack_ensure_no_arg --auth "${mongodHackedArgs[@]}"
if [ "$MONGO_INITDB_ROOT_USERNAME" ] && [ "$MONGO_INITDB_ROOT_PASSWORD" ]; then
_mongod_hack_ensure_no_arg_val --replSet "${mongodHackedArgs[@]}"
fi
# "BadValue: need sslPEMKeyFile when SSL is enabled" vs "BadValue: need to enable SSL via the sslMode flag when using SSL configuration parameters"
tlsMode='disabled'
if _mongod_hack_have_arg '--tlsCertificateKeyFile' "$@"; then
tlsMode='allowTLS'
elif _mongod_hack_have_arg '--sslPEMKeyFile' "$@"; then
tlsMode='allowSSL'
fi
# 4.2 switched all configuration/flag names from "SSL" to "TLS"
if [ "$tlsMode" = 'allowTLS' ] || mongod --help 2>&1 | grep -q -- ' --tlsMode '; then
_mongod_hack_ensure_arg_val --tlsMode "$tlsMode" "${mongodHackedArgs[@]}"
else
_mongod_hack_ensure_arg_val --sslMode "$tlsMode" "${mongodHackedArgs[@]}"
fi
if stat "/proc/$$/fd/1" > /dev/null && [ -w "/proc/$$/fd/1" ]; then
# https://github.com/mongodb/mongo/blob/38c0eb538d0fd390c6cb9ce9ae9894153f6e8ef5/src/mongo/db/initialize_server_global_state.cpp#L237-L251
# https://github.com/docker-library/mongo/issues/164#issuecomment-293965668
_mongod_hack_ensure_arg_val --logpath "/proc/$$/fd/1" "${mongodHackedArgs[@]}"
else
initdbLogPath="$(_dbPath "$@")/docker-initdb.log"
echo >&2 "warning: initdb logs cannot write to '/proc/$$/fd/1', so they are in '$initdbLogPath' instead"
_mongod_hack_ensure_arg_val --logpath "$initdbLogPath" "${mongodHackedArgs[@]}"
fi
_mongod_hack_ensure_arg --logappend "${mongodHackedArgs[@]}"
pidfile="${TMPDIR:-/tmp}/docker-entrypoint-temp-mongod.pid"
rm -f "$pidfile"
_mongod_hack_ensure_arg_val --pidfilepath "$pidfile" "${mongodHackedArgs[@]}"
"${mongodHackedArgs[@]}" --fork
mongo=( mongo --host 127.0.0.1 --port 27017 --quiet )
# check to see that our "mongod" actually did start up (catches "--help", "--version", MongoDB 3.2 being silly, slow prealloc, etc)
# https://jira.mongodb.org/browse/SERVER-16292
tries=30
while true; do
if ! { [ -s "$pidfile" ] && ps "$(< "$pidfile")" &> /dev/null; }; then
# bail ASAP if "mongod" isn't even running
echo >&2
echo >&2 "error: $originalArgOne does not appear to have stayed running -- perhaps it had an error?"
echo >&2
exit 1
fi
if "${mongo[@]}" 'admin' --eval 'quit(0)' &> /dev/null; then
# success!
break
fi
(( tries-- ))
if [ "$tries" -le 0 ]; then
echo >&2
echo >&2 "error: $originalArgOne does not appear to have accepted connections quickly enough -- perhaps it had an error?"
echo >&2
exit 1
fi
sleep 1
done
if [ "$MONGO_INITDB_ROOT_USERNAME" ] && [ "$MONGO_INITDB_ROOT_PASSWORD" ]; then
rootAuthDatabase='admin'
"${mongo[@]}" "$rootAuthDatabase" <<-EOJS
db.createUser({
user: $(_js_escape "$MONGO_INITDB_ROOT_USERNAME"),
pwd: $(_js_escape "$MONGO_INITDB_ROOT_PASSWORD"),
roles: [ { role: 'root', db: $(_js_escape "$rootAuthDatabase") } ]
})
EOJS
fi
export MONGO_INITDB_DATABASE="${MONGO_INITDB_DATABASE:-test}"
echo
for f in /docker-entrypoint-initdb.d/*; do
case "$f" in
*.sh) echo "$0: running $f"; . "$f" ;;
*.js) echo "$0: running $f"; "${mongo[@]}" "$MONGO_INITDB_DATABASE" "$f"; echo ;;
*) echo "$0: ignoring $f" ;;
esac
echo
done
"${mongodHackedArgs[@]}" --shutdown
rm -f "$pidfile"
echo
echo 'MongoDB init process complete; ready for start up.'
echo
fi
# MongoDB 3.6+ defaults to localhost-only binding
haveBindIp=
if _mongod_hack_have_arg --bind_ip "$@" || _mongod_hack_have_arg --bind_ip_all "$@"; then
haveBindIp=1
elif _parse_config "$@" && jq --exit-status '.net.bindIp // .net.bindIpAll' "$jsonConfigFile" > /dev/null; then
haveBindIp=1
fi
if [ -z "$haveBindIp" ]; then
# so if no "--bind_ip" is specified, let's add "--bind_ip_all"
set -- "$@" --bind_ip_all
fi
unset "${!MONGO_INITDB_@}"
fi
rm -f "$jsonConfigFile" "$tempConfigFile"
exec "$@"
mongo/js-yaml.min.js 0 → 100644
View file @ f47954a6
This diff is collapsed.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment