# dtu-enote

This is the setup we want to end up with.
```
http:letsencrypt-enote.compute.dtu.dk    used only to renew certs

https://enote.compute.dtu.dk             a landing site with just a static html file

https://quiz.compute.dtu.dk              shared quiz for all courses
https://file-admin.compute.dtu.dk        shared File Admin for all courses
https://stackedit.compute.dtu.dk         shared Stack Edit for all courses

https://01005.compute.dtu.dk/            course website (rails)
https://01005.compute.dtu.dk/askbot      askbot just for 01005
https://01005.compute.dtu.dk/sharelatex  sharelatex just for 01005
```
To simulate that setup I would like to end up with
```
http:letsencrypt-enote.compute.dtu.dk       used only to renew certs

https://testlanding.compute.dtu.dk          a landing site with just a static html file

https://testsite1.compute.dtu.dk/           course website (rails)
https://testsite1.compute.dtu.dk/askbot     askbot site just for "testsite1".
```
where each domain is its own container.

`./run.sh` sets that up, but where https is missing.

The idea is that `letsencrypt-enote` shares an volume, containing the SSL certs, with the reverse proxy container, so the certs can be updated and in worst case also have to reload the reverse proxy container.

Maybe `quiz` wouldn't be in the reverse proxy, as it might have deep letsencrypt integrations. Ask Iain.


# Organization and Setup

This is the root project for enote.  It consists of a reverse proxy setup and a number of child projects (in separate repos) to which the main server proxies requests.  Each project (and the nginx proxy frontend) are run in separate containers.

dtu-enote/components
dtu-enote/components/askbot
dtu-enote/components/stackedit
dtu-enote/components/file-manager
dtu-enote/components/course-website
dtu-enote/components/couchdb
dtu-enote/components/dtu-quiz


```
course-website    -> dtu-enote-website
docker-askbot     -> dtu-enote-askbot
docker-cn-dumper  -> dtu-enote-cndumper
docker-couchdb    -> dtu-enote-couchdb
docker-get-pdf    -> dtu-enote-getpdf
docker-mongodb    -> dtu-enote-mongodb
docker-sharelatex -> dtu-enote-sharelatex
stackedit         -> dtu-enote-stackedit
dtu-core          -> dtu-enote-core
quiz-parser       -> dtu-quiz-parser
pdf.js            -> dtu-enote-pdfjs


repos=("dtu-enote-website"
"dtu-enote-askbot"
"dtu-enote-cndumper"
"dtu-enote-couchdb"
"dtu-enote-getpdf"
"dtu-enote-mongodb"
"dtu-enote-sharelatex"
"dtu-enote-stackedit"
"dtu-enote-core"
"dtu-quiz-parser"
"dtu-enote-pdfjs"
"dtu-enote"
"dtu-quiz"
"dtu-enote-core" )

for r in ${repos[@]}
do
  echo $r
  git clone -b master https://github.com/dtu-compute/$r
  pushd $r
  git fetch origin
  popd
done

for r in ${repos[@]}
do
  echo $r
  pushd $r
  git fetch origin 
  git log --left-right --graph --cherry-pick --oneline origin/2016...origin/devel
  popd
done


pushd components
branch=$(git symbolic-ref --short HEAD)
git clone -b $branch https://github.com/dtu-compute/dtu-enote-sharelatex sharelatex
git clone -b $branch https://github.com/dtu-compute/dtu-enote-installer
git clone -b $branch https://github.com/dtu-compute/dtu-enote-cndumper cndumper
git clone -b $branch https://github.com/dtu-compute/dtu-enote-couchdb couchdb
git clone -b $branch https://github.com/dtu-compute/dtu-enote-website course-website
git clone -b $branch https://github.com/dtu-compute/dtu-enote-mongodb mongodb
git clone -b $branch https://github.com/dtu-compute/dtu-enote-core dtu-core
git clone -b $branch https://github.com/dtu-compute/dtu-quiz-nginx dtu-quiz-nginx
git clone -b $branch https://github.com/dtu-compute/dtu-enote-landing landing
git clone -b $branch https://github.com/dtu-compute/dtu-enote-letsencrypt letsencrypt
git clone -b $branch https://github.com/dtu-compute/dtu-enote-course-website
git clone -b $branch https://github.com/dtu-compute/dtu-enote-stackedit
git clone -b $branch https://github.com/dtu-compute/dtu-enote-docker-couchdb couchdb
git clone -b $branch https://github.com/dtu-compute/dtu-enote-docker-getpdf get-pdf

find . -name .git -type d -prune 
find components -name .git -type d -prune -exec git --git-dir={}/ --work-tree=$PWD/{}/.. pull origin devel \;

find components -name .git -type d -prune -exec git --git-dir={}/ --work-tree=$PWD/{}/.. status \;


for i in components/*/.git; do ( echo $i; cd $i/..; git pull; ); done

popd
```


# Building

```
alias dep='bundle exec rake deploy:all'
```

```
./run.sh -disable:sharelatex -disable:askbots -disable:getpdf
```


# shared folders

```
chcon -Rt svirt_sandbox_file_t /filemanager/
````

echo "<html><body>hi</body></html>" > /foo/index.html


# Debugging

## Shell Into Containsers

```
docker exec -i -t landing /bin/bash 
docker exec -i -t letsencrypt /bin/bash 
docker exec -i -t proxy /bin/bash 
docker exec -i -t couchdb /bin/bash 
docker exec -i -t stackedit /bin/bash 
docker exec -i -t mongodb /bin/bash 
docker exec -i -t website-00000 /bin/bash
docker exec -i -t website-01005 /bin/bash
docker exec -i -t website-01006 /bin/bash
docker exec -i -t website-02402 /bin/bash
docker exec -i -t file-manager /bin/bash
docker exec -i -t get-pdf /bin/bash
docker exec -i -t askbot-00000 /bin/bash
docker exec -i -t askbot-01005 /bin/bash
docker exec -i -t quiz /bin/bash
docker exec -i -t quiz-db /bin/bash
docker exec -i -t mongodb-sharelatex /bin/bash
docker exec -i -t sharelatex-01005 /bin/bash


## View Logs

### Container Logs

```
docker logs file-manager 
docker logs quiz_1
```


### Logs from Applications
```
docker exec -i -t course-website-00000 cat /dtu-course/log/production.log
docker exec -i -t website-01005 cat /dtu-course/log/production.log
docker exec -i -t website-02402 cat /dtu-course/log/production.log
docker exec -i -t website-02323 cat /dtu-course/log/production.log
docker exec -i -t file-manager cat /file-manager/app/log/production.log
docker exec -i -t quiz cat /dtu-quiz/app/log/production.log | tail -500
docker exec -i -t quiz cat /dtu-quiz/app/log/populate.log
docker exec -i -t quiz cat /file-manager/app/log/production.log
docker exec -i -t askbot-01005 cat /data/log/askbot.log

docker exec -i -t letsencrypt cat /var/log/nginx/error.log

docker exec -i -t letsencrypt cat/var/log/nginx/access.log 

docker exec -i -t sharelatex-00000 cat /var/log/sharelatex/web.log
```


## Populate redis with identity information

```
docker exec -i -t quiz bundle exec rake populate
docker exec -i -t dtuenotedevel_quiz bundle exec rake populate

```


## DB Operations

### Clean couchdb

```
docker exec -i -t couchdb /clean-db.sh
rm -rf /usr/local/var/lib/couchdb/*
```

### Shutdown DBs

```
docker exec -i -t couchdb /shutdown.sh
docker exec -i -t mongodb /shutdown.sh
docker exec -ti sharelatex-db killall mongod
```


more /var/log/nginx/access.log
tail /var/log/nginx/access.log

# find rails failures
docker logs course_website | grep -A 5 -B 5 "\(ERROR\|FATAL\)"
```

## CRON


cp crontab /etc/cron.d/populate-cron
chmod 0644 /etc/cron.d/populate-cron
touch /enote/vol/log/quiz/populate.log
crontab -e

## Enabling and disabling components

```
./run.sh disable:sharelatex disable:letsencrypt disable:askbot
```


## Adding new Courses

1. Update the courses. Add the new one with `enabled: false` in `courses.yaml`

```
/enote/vol/config/courses.yaml
/enote/vol/config/cn-dumper.yaml
```

2. (Optional)
`run.sh`

3. Create new certs

```
./make-certs.sh .compute.dtu.dk
```

Note: the URL suffix is required

4. Update the courses.yaml to have `enabled:true`

5. Restart

```
run.sh
```

## Re-uploading raw Web site files to couchdb

1. Make sure node is installed

```
curl --silent --location https://rpm.nodesource.com/setup_6.x | bash -
yum install -y nodejs
```

2. Set up NPM

```
pushd components/stackedit
npm install
```

3. Upload files

```
 ./couchdb_import.sh 01006 /enote/vol/website-raw/01006
 ./couchdb_import.sh 00000 /enote/vol/website-raw/00000
 ./couchdb_import.sh 02402 /enote/vol/website-raw/02402
```

## Updating course Web site

```
docker exec -i -t website-00000 bundle exec rake update:all
docker exec -i -t website-00000 bundle exec rake update:compile_pages
docker exec -i -t website-00000 bundle exec rake update:podcasts
docker exec -i -t website-02323 bundle exec rake update:all
```

```
docker exec -i -t quiz bundle exec rake dtuquiz:dump_course['00000']
```


### Errors

This can happen:

```
There were too many requests of a given type :: Error creating new cert :: Too many certificates already issued for: dtu.dk
Please see the logfiles in /var/log/letsencrypt for more details.
askbot-00000-enote.compute.dtu.dk
```

Helpful commands to investigate:

```
docker exec -i -t letsencrypt ls /var/log/letsencrypt
docker exec -i -t letsencrypt /bin/bash 
```


### Building and Pushing Sharelatex Containers

We use two docker hub containers: one for the base sharelatex and one for the enote-specific sharelatex.

#### sharelatex-enote-base

We build our own version of the base sharelatex image to ensure that it doesn't change (and, initially at least, to fix the tlmgr [bug](https://github.com/sharelatex/sharelatex-docker-image/issues/77) ).  The sharelatex image is itself based on a base image with another layered on top.  To build it it the `enote` project root:

```
pushd components
git clone https://github.com/sharelatex/sharelatex-docker-image.git
popd
./build-sharelatex-enote-base.sh
```

This will concatenate the two `Dockerfile`s describing the sharelatex image, build it, and publish it. `docker login` may be required in order to push.

#### sharelatex-enote

After the base is build, you can build the enote-specific image:

```
pushd components/sharelatex
./build-sharelatex-enote.sh
```

enote itself will simply use the `sharelatex-enote` image.