- What is the Data team server?
- Who controls the server?
- What lives on the server?
- How do I access the server?
- How do I add or remove users from the server?
The Data team server is a remote machine that we use to host various tools and services. It is a virtual machine (VM) running on an on-premise server within the Cook County building.
A sample of the types of things we use the server for include:
- Run RStudio and JupyterLab instances with more compute than we have on our laptops
- Schedule cron jobs to perform ETL tasks
- Run web apps using Docker
These examples are intended to give you a high-level overview of the types of tools we host on the server. For more detail on the tools that are available on the server, see: What lives on the server?
The server is owned and maintained collectively by the Data team staff, but only a small group of Senior Data Scientists have superuser permissions (these users are sudoers in Linux terminology). If you need access to superuser permissions (to run commands that require sudo, for example), contact a Senior Data Scientist.
The hardware that the server runs on is controlled by the CCAO's internal IT. If you need hardware debugging/upgrades or a hard reboot, contact the IT Admins.
The following is a list of tools and services that live on the server as of summer 2023:
| Path | Description | Repo | Run By |
|---|---|---|---|
/home/<username> |
Home directories for each member of the data team | ||
/home/shiny-server/services |
Home directory for various apps that run on the server | ||
/home/shiny-server/services/service-spark-iasworld |
Service that extracts data from the iasWorld platform and loads it into AWS S3 | service-spark-iasworld |
cron |
/home/shiny-server/services/service-socrata-ingress-agent |
Service that extracts data from AWS Athena and loads it into the Socrata data portal | service-socrata-ingress-agent |
systemd |
/opt/aws/amazon-cloudwatch-agent/etc |
Service to grab local Socrata logs and move them to AWS CloudWatch | service-socrata-ingress-agent |
systemd |
/home/shiny-server/services/service-watchtower |
Service that keeps Docker images up to date on the server | service-watchtower |
docker |
/home/shiny-server/services/service-nginx |
Service that directs web traffic to other services | service-nginx |
docker |
/home/shiny-server/services/service-shiny-proxy |
Service that runs other services using Docker | service-shiny-proxy |
docker |
/home/shiny-server/services/service-jupyterhub |
JupyterHub instance for running notebooks | service-jupyterhub |
systemd |
/home/shiny-server/services/api-res-avm |
REST API for getting predicted values from our residential model | api-res-avm |
docker |
/var/spool/cron/crontabs/shiny-server |
Cron job configuration | cron | |
/etc/systemd/system/ |
systemd service configurations | systemd |
If you add a new tool or service to the server, add it to this list and bump the date listed above.
Ask a Senior Data Scientist to create a user account for you on the server if you don't have one already.
Once you have a user account, you can connect the server via SSH:
This will prompt you to enter the password that was created for your account.
Once you've logged in for the first time, you can change your password by running the passwd command; you can also configure your account to log in using an SSH key instead of a password if you prefer.
See this document for details.