diff --git a/docs/indexing.md b/docs/indexing.md index 733d17c4..093c465a 100644 --- a/docs/indexing.md +++ b/docs/indexing.md @@ -95,10 +95,10 @@ Takes 6-7 hours to complete. If needed, [use the SCSB API to request new full dump records from the system to be generated into the SCSB bucket](./scsb/request_full.md). -If needed, [pull the most recent SCSB full dump records into dump files](./scsb/dump_files.md). +Then, if needed, [pull the most recent SCSB full dump records into dump files](./scsb/dump_files.md). This is only necessary if the most recent "Full Partner ReCAP Records" is missing files, or if the monthly process hasn't run for a while and there is no recent event of this type. -Note that this process takes 24 hours, and you can't deploy in the middle of the process. +Note that this process takes 12 hours, and you can't deploy in the middle of the process. Once the files are all downloaded and processed, index them with diff --git a/docs/scsb/dump_files.md b/docs/scsb/dump_files.md index 853bbd80..45c60fb4 100644 --- a/docs/scsb/dump_files.md +++ b/docs/scsb/dump_files.md @@ -9,5 +9,5 @@ CTRL+b d (to detach from tmux) ``` This kicks off an import job which will return immediately. This can be monitored in [sidekiq busy queue](https://bibdata.princeton.edu/sidekiq/busy) or [sidekiq waiting queue](https://bibdata.princeton.edu/sidekiq/queues/default) -Takes 24-25 hours to complete. As they download and unpack they will be placed +Takes 11-12 hours to complete. As they download and unpack they will be placed in `/tmp/updates/` and as they are processed they will be moved to `/data/bibdata_files/scsb_update_files/`; you can follow the progress by listing the files in these directories. You can also find the most recent Full Partner ReCAP Records from [the events page](https://bibdata.princeton.edu/events), and look at the dump files in its json. Be sure not to deploy bibdata in the middle of this job, or else the job will have to start all over again from the beginning. \ No newline at end of file