File tree Expand file tree Collapse file tree 1 file changed +11
-9
lines changed Expand file tree Collapse file tree 1 file changed +11
-9
lines changed Original file line number Diff line number Diff line change 3
3
//
4
4
// Example usage:
5
5
//
6
- // $ span-crossref-sync \
7
- // -p zstd \ # compress program
8
- // -P feed-1- \ # file prefix (to separate different runs)
9
- // -i d \ # interval (daily)
10
- // -verbose \ # verbose
11
- // -t 30m \ # timeout
12
- // -s 2022-01-01 \ # start
13
- // -e 2023-05-01 \ # end (default: yesterday)
14
- // -c /data/finc/crossref/ # cache dir
6
+ // $ span-crossref-sync \
7
+ // -p zstd \ # compress program
8
+ // -P feed-1- \ # file prefix (to separate different runs)
9
+ // -i d \ # interval (daily)
10
+ // -verbose \ # verbose
11
+ // -t 30m \ # timeout
12
+ // -s 2022-01-01 \ # start
13
+ // -e 2023-05-01 \ # end (leave out for default: yesterday)
14
+ // -c /data/finc/crossref/ # cache dir
15
15
//
16
16
// Space requirements: One day yields about 1M update docs, or a ~2GB
17
17
// compressed file. A year equates to about 800G of compressed data.
21
21
// snapshot.
22
22
//
23
23
// Data point: https://github.com/miku/filterline#data-point-crossref-snapshot
24
+ //
25
+ // As of 02/2024 we have 768 files (for "feed-1-") using 2.1TB (zstd, est. 12TB uncompressed).
24
26
package main
25
27
26
28
import (
You can’t perform that action at this time.
0 commit comments