Manual Deployment Documentation

How to configure the instance

Instance type: Ubuntu 18.04 LTS

Step 1: Install MySQL

• https://linuxize.com/post/how-to-install-mysql-on-ubuntu-18-04/
• User : root => factbounty
• CREATE DATABASE factbounty_dev;
• CREATE DATABASE factbounty_prod;
• Solves host connection issue: https://support.plesk.com/hc/en-us/articles/360005493133-Unable-to-register-remote-MySQL-server-Host-is-not-allowed-to-connect-to-this-MySQL-server
• https://devanswers.co/cant-connect-mysql-server-remotely/

Step 2: Install Elastic Search

• https://sundog-education.com/elasticsearch/
• Hosted: http://35.200.224.48:9200/
• Check status: systemctl status elasticsearch.service

Step 3: Install Kibana

• https://www.techrepublic.com/article/how-to-install-elasticsearch-on-ubuntu-server-18-04
• Hosted: http://35.200.224.48:5601/
• Check status: systemctl status kibana.service

Step 4: Set up crawlers project

• Install Git: sudo apt install git
• Clone repo and cd to crawlers
• Install Python 3 : http://ubuntuhandbook.org/index.php/2017/07/install-python-3-6-1-in-ubuntu-16-04-lts/
• Install PIP3: https://linuxize.com/post/how-to-install-pip-on-ubuntu-18.04/
• pip install virtualenv
• virtualenv -p python3 venv
• source venv/bin/activate
• pip3 install -r requirements.txt
• Add bind address to scrapyd.conf: bind_address = 0.0.0.0
• [scrapyd]
• nohup scrapyd &
• bind_address = 0.0.0.0
• Change elastic search url in Setting.py
• Hosted: http://35.200.224.48:6800/

Step 5: Set up flask project

• Create Build of Client project
• Add .env file
• source venv/bin/activate
• pip3 install -r requirements.txt
• https://stackoverflow.com/questions/51919567/flask-on-google-compute-engine-cant-reach-from-outside-browser
• flask run --host=0.0.0.0 --port=5000
• nohup flask run --host=0.0.0.0 --port=5000 &
• Hosted: http://35.200.224.48:5000/

note:

• if name == 'main': app.run(host='0.0.0.0', port=5000)

Step 6: Build Client

• sudo apt-get install nodejs
• Remove dir: rm -r build

Keep Process running:

• nohup [your command] &
• https://superuser.com/questions/1110574/keeping-a-java-app-running-when-ssh-is-closed-on-google-cloud
• To stop: https://stackoverflow.com/questions/17385794/how-to-get-the-process-id-to-kill-a-nohup-process