Changeset - af5fec11ff33
[Not reviewed]
2 5 0
James Polley - 7 years ago 2017-10-04 09:43:01
jp@jamezpolley.com
Utilise multi-stage docker build for consistent dev/prod base

* developers can use --target symposion_dev to get a responsive site
that reads from their homedir and reacts to changed files instantly
* without a specified target the default is to build the prod image,
which is identical except for running uwsgi instead of the django
built-in server
* Enable debug when running in a developer's test environment
* Remove the makemigrations script and dockerfile
7 files changed with 18 insertions and 39 deletions:
0 comments (0 inline, 0 general)
.dockerignore
Show inline comments
 
.git
 
ve
 
symposion-fixtures
 

	
README.rst
Show inline comments
...
 
@@ -3,132 +3,133 @@ symposion_app
 

	
 
.. contents::
 

	
 
At this time, considerations have not been made to have the django project run
 
without additional infrastructure.
 

	
 
This can be configured in the future by switching the default
 
DEFAULT_FILE_STORAGE django default, and properly configuring django to load
 
subsequent configuration to switch this back to GCS on run in our testing
 
and production environments.
 

	
 
Login is a bit more complicated, the default flow redirects you out to a SAML
 
server, you can probably work something out with testshib.   But if you go to
 
/admin and log in, that will persist into /dashboard.
 

	
 

	
 
Required Configuration
 
----------------------
 

	
 
GCS
 
~~~
 

	
 
1. Set the bucket in settings.py
 
1. Store your Service Account JSON in a file
 
1. Set GOOGLE_APPLICATION_CREDENTIALS to your account file
 
1. Set GCS_BUCKET to your bucket name
 

	
 
SAML2
 
~~~~~
 

	
 
Please familiarise yourself with Shibboleth configuration and the general
 
flow of how to register a SP with an IDP.
 

	
 
If you send the SP metadata statically, validity time must be removed.
 

	
 
You will also need to register your IDP metadata here either statically or
 
remotely updated.
 

	
 
You may consider testing with testshib.
 

	
 
Configure signing and encryption keys, and configure them in the settings.py
 

	
 
Running a dev instance in Docker
 
--------------------------------
 

	
 
Assuming you have docker installed and working on your machine::
 
    ./make_dev_container.sh
 

	
 
will build you a container and run through the initial setup steps.
 
The last stage interactively creates a superuser account: you'll need
 
to interact with this for it to finish.
 

	
 
Once this has completed, you can hit http://localhost:28000/admin to
 
log into the admin interface. Once you're logged in,
 
http://localhost:28000 will take you to the dashboard.
 

	
 
Note that when you do this you're logged in as a superuser, so what
 
you see will be different from what a normal user will see.
 

	
 
Making migrations
 
~~~~~~~~~~~~~~~~~
 

	
 
If you make changes to the data model, you'll need to run "manage.py
 
makemigrations" to create a matching migration file. If you're on a
 
mac, or a system without python3, this can be difficult. In such a
 
case, the ``makemigrations.sh`` script takes advantaged of a docker
 
container that's slightly modified, and runs the makemigration action
 
on the files in your working directory.
 
mac, or a system without python3, this can be difficult.
 

	
 
In such a case, you can use the above script to make and run a dev
 
container; then::
 
    docker exec -it symposion ./manage.py makemigrations
 

	
 

	
 
Running a dev instance in a VirtualEnv
 
--------------------------------------
 

	
 
Not all things are lovely, so we use constraints to force the versions we
 
we wish for without having to do anything ugly.  This may require a newer
 
version of pip than is packaged with distros virtualenv.
 

	
 
Note that this application is python 3 only so you must create your virtualenv
 
with a python3 interpreter.
 

	
 
- ``python3 -m venv venv``
 
- ``source ./venv/bin/activate``
 
- ``pip install -c constraints.txt -r requirements.txt``
 
- ``pip install -c constraints.txt -r vendored_requirements.txt``
 

	
 
Once your dev instance is up and running
 
----------------------------------------
 

	
 
Pre-post-start configuration
 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Now that we are ready to start the app, we need to do initial setup, start
 
it, perform voodoo string magic, run more database jingling voodoo, and then
 
we can start it up and start looking around and customising the application.
 

	
 
``./manage.py migrate``
 
``./manage.py runserver``
 

	
 
Now we hit up the admin interface, find `symposion_proposals` `proposalkind`
 
and add `Talk` slug `talk`, `Tutorial`:`tutorial`, and `Miniconf`:`miniconf`.
 

	
 
Shut down and now run:
 
``./manage.py populate_inventory``
 

	
 
Now you can run the system and see how it goes.
 

	
 
Admin tasks
 
-----------
 

	
 
Admin Credentials
 
~~~~~~~~~~~~~~~~~
 

	
 
To create a local administrator account run:
 

	
 
``./manage.py createsuperuser``
 

	
 
and enter your username, email and password. This can then be used to login to: (http://localhost:8000/admin).
 

	
 
Base Data
 
~~~~~~~~~
 

	
 
On initial creation and whenever you reset the database you must reload the
 
basic data. This data is stored in the /fixtures directory. You can load this with:
 

	
 
``./manage.py loaddata ./fixtures/{conference,proposal_base,sites,sitetree}.json``
 

	
 
And to load the base admin users:
 

	
 
``./manage.py loaddata ./fixtures/admin_users.json``
 

	
 
admin1:Inq4JVQyQvWnqXDI
 

	
docker/Dockerfile
Show inline comments
 
FROM python:3.6
 

	
 
FROM python:3.6 as symposion_base
 

	
 
RUN set -ex \
 
    && apt-get update
 

	
 
RUN set -ex \
 
    && buildDeps=' \
 
        libmysqlclient-dev \
 
        libffi-dev \
 
        libfreetype6-dev \
 
        libjpeg-dev \
 
        libwebp-dev \
 
        libpng-dev \
 
        liblcms2-dev \
 
        zlib1g-dev \
 
        libmemcached-dev \
 
        libsasl2-dev \
 
    ' \
 

	
 
    && apt-get install -y git xmlsec1 libmysqlclient18 \
 
    && apt-get install -y $buildDeps --no-install-recommends \
 
    && rm -rf /var/lib/apt/lists/*
 

	
 
RUN set -ex \
 
    && pip install uwsgi
 

	
 
COPY constraints.txt requirements.txt /reqs/
 

	
 
RUN set -ex \
 
    && pip install --no-cache-dir -r /reqs/requirements.txt -c /reqs/constraints.txt \
 
    && apt-get purge -y --auto-remove $buildDeps \
 
    && rm -rf /usr/src/python ~/.cache
 

	
 
COPY . /app/symposion_app
 

	
 
WORKDIR /app/symposion_app
 
RUN set -x \
 
    && pip install -r vendored_requirements.txt -c /reqs/constraints.txt
 
RUN set -x \
 
    && DJANGO_SECRET_KEY=1234 STRIPE_PUBLIC_KEY=1234 STRIPE_SECRET_KEY=1234 \
 
       DATABASE_URL="sqlite:////dev/null" \
 
       python manage.py collectstatic --noinput -l -v 0
 

	
 
FROM symposion_base as symposion_dev
 
VOLUME /app/symposion_app
 
CMD ["./manage.py", "runserver", "-v3", "0.0.0.0:8000"]
 

	
 
FROM symposion_base as symposion_prod
 
CMD ["/usr/local/bin/uwsgi", "--http-socket", "0.0.0.0:8000", "--wsgi-file", "pinaxcon/wsgi.py"]
docker/Dockerfile.makemigrations
Show inline comments
 
deleted file
docker/laptop-mode-env
Show inline comments
 
DJANGO_SECRET_KEY=5CEA51A5-A613-4AEF-A9FB-D0A57D77C13B
 
STRIPE_PUBLIC_KEY=5CEA51A5-A613-4AEF-A9FB-D0A57D77C13B
 
STRIPE_SECRET_KEY=5CEA51A5-A613-4AEF-A9FB-D0A57D77C13B
 
GCS_BUCKET=5CEA51A5-A613-4AEF-A9FB-D0A57D77C13B
 
GOOGLE_APPLICATION_CREDENTIALS=/dev/null
 
DATABASE_URL=sqlite:////tmp/symposion.sqlite
 
SYMPOSION_DEV_MODE=LAPTOP
...
 
\ No newline at end of file
 
SYMPOSION_DEV_MODE=LAPTOP
 
SYMPOSION_APP_DEBUG=1
...
 
\ No newline at end of file
make_dev_container.sh
Show inline comments
 
#!/bin/bash -x
 

	
 
CONTAINER_NAME=${1:-symposion_app}
 

	
 
docker image build -f docker/Dockerfile -t ${CONTAINER_NAME} .
 
docker image build -f docker/Dockerfile -t ${CONTAINER_NAME} --target symposion_dev .
 
docker container stop symposion
 
docker container rm symposion
 
docker container create --env-file docker/laptop-mode-env -p 28000:8000 --name symposion ${CONTAINER_NAME}
 
docker container create --env-file docker/laptop-mode-env -p 28000:8000 -v $(pwd):/app/symposion_app --name symposion ${CONTAINER_NAME}
 
docker container start symposion
 
docker exec symposion ./manage.py migrate
 
docker exec symposion ./manage.py loaddata ./fixtures/*.json
 
docker exec symposion ./manage.py populate_inventory
 
docker exec symposion ./manage.py create_review_permissions
 
docker exec symposion ./manage.py loaddata ./fixtures/miniconf-fixtures/*.json
 
if [ -e ./symposion-fixtures ]; then
 
    pushd ./symposion-fixtures
 
    ./load_chunks_local.sh
 
    popd
 
else
 
    docker exec -it symposion ./manage.py createsuperuser --username root --email root@example.com
 
fi
 

	
 
set +x
 
echo "Now you can log into http://localhost:28000/admin"
 
echo "Username: root      Password: the one you just typed twice"
 
echo "If you need to test as a non-admin user, create one at"
 
echo "http://localhost:28000/admin/auth/user/add/ - then log out"
 
echo "and log back in at http://localhost:28000"
 

	
makemigrations.sh
Show inline comments
 
deleted file
0 comments (0 inline, 0 general)