Skip to content

Update Google Custom Search (GCS) and add 2024Q4 report #148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 35 commits into from
Jan 9, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
6cfacd2
add/update processed gcs data
TimidRobot Dec 13, 2024
f05b713
add totals_by_country and totals_by_langauage
TimidRobot Dec 13, 2024
3f5a441
improve argument and error handling
TimidRobot Dec 13, 2024
525aa37
improve argument and error handling
TimidRobot Dec 13, 2024
62e6f9e
improve argument and error handling
TimidRobot Dec 13, 2024
b1a0fba
use supplied logger and improve info message
TimidRobot Dec 13, 2024
f314773
refactor to include updates to flow (--enable-save, --enable-git), hi…
TimidRobot Dec 13, 2024
0713b37
add plot_totals_by_product
TimidRobot Dec 18, 2024
ba7093f
improve Google Custom Search (GCS) fetch accuracy with quotes and lin…
TimidRobot Dec 18, 2024
ce68a8a
update data following accuracy improvements
TimidRobot Dec 19, 2024
2c0f4fa
Merge branch 'improve-gcs-accuracy' into moar-gcs
TimidRobot Dec 19, 2024
a73aa36
Refactor for clarity and to be more "pythonic"
TimidRobot Dec 19, 2024
955a29e
separate caption text and entry text
TimidRobot Dec 19, 2024
24741cf
add GCS intro and references
TimidRobot Dec 19, 2024
99bc996
rename processed data sets. add current, old, retired. rmove top 25
TimidRobot Dec 21, 2024
713b0c4
update reporting plot styles and add gcs current, old, retired
TimidRobot Dec 21, 2024
c2e9a4a
re-enable plots
TimidRobot Dec 21, 2024
2c61cf5
update processed data
TimidRobot Dec 23, 2024
00c8814
refactor GCS report
TimidRobot Dec 23, 2024
0d5fffd
Merge branch 'main' into moar-gcs
TimidRobot Dec 23, 2024
f9bf303
update processed data with more accurate gcs data
TimidRobot Dec 23, 2024
a117c6a
fix spelling mistake
TimidRobot Dec 23, 2024
711ecb6
improve naming and add plots
TimidRobot Dec 23, 2024
a734985
Merge branch 'main' into moar-gcs
TimidRobot Jan 6, 2025
5a1e162
add and fix support for specifying quarter
TimidRobot Jan 6, 2025
52e1a02
rename report to be more generic
TimidRobot Jan 6, 2025
ae0433f
add support for specifying quarter and usage section
TimidRobot Jan 6, 2025
c7a763d
rename functions for easier sorting
TimidRobot Jan 6, 2025
5ed2a7d
sort functions
TimidRobot Jan 6, 2025
ee3f5f8
move plotting code to shared library
TimidRobot Jan 6, 2025
000b7a3
remove extra space
TimidRobot Jan 6, 2025
c757504
add 2024Q4 report
TimidRobot Jan 6, 2025
0647329
Impove terms and wording
TimidRobot Jan 9, 2025
dbf9ef1
provide more context for Approved for Free Cultural Works
TimidRobot Jan 9, 2025
f8b3cca
fix data paths
TimidRobot Jan 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions data/2024Q4/2-process/gcs_product_totals.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
"CC legal tool product","Count"
"Licenses version 4.0","1132000000"
"Licenses version 3.0","15289017400"
"Licenses version 2.x","18343329641"
"Licenses version 1.0","1918709000"
"CC0 1.0","30500000"
"Public Domain Mark 1.0","8180000"
"Certification 1.0 US","47000000"
4 changes: 4 additions & 0 deletions data/2024Q4/2-process/gcs_status_combined_totals.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
"CC legal tool","Count"
"Latest","1170680000"
"Prior","33776380800"
"Retired","1821675241"
9 changes: 9 additions & 0 deletions data/2024Q4/2-process/gcs_status_latest_totals.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
"CC legal tool","Count"
"CC BY 4.0","324000000"
"CC BY-NC 4.0","132000000"
"CC BY-NC-ND 4.0","132000000"
"CC BY-NC-SA 4.0","58000000"
"CC BY-ND 4.0","322000000"
"CC BY-SA 4.0","164000000"
"PDM 1.0","8180000"
"CC0 1.0","30500000"
7 changes: 7 additions & 0 deletions data/2024Q4/2-process/gcs_status_prior_totals.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
"CC legal tool","Count"
"CC BY","13232393000"
"CC BY-NC","3478532000"
"CC BY-NC-ND","2239689000"
"CC BY-NC-SA","1549258800"
"CC BY-ND","6724112000"
"CC BY-SA","6552396000"
11 changes: 11 additions & 0 deletions data/2024Q4/2-process/gcs_status_retired_totals.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
"CC legal tool","Count"
"CC DEVNATIONS","241"
"CC NC","111130000"
"CC NC-SA","42045000"
"CC ND","533000000"
"CC ND-NC","105730000"
"CC SA","240070000"
"CC NC-SAMPLING+","50000000"
"CC SAMPLING","284800000"
"CC SAMPLING+","407900000"
"CC PUBLICDOMAIN","47000000"
26 changes: 0 additions & 26 deletions data/2024Q4/2-process/gcs_top_25_tools.csv

This file was deleted.

243 changes: 243 additions & 0 deletions data/2024Q4/2-process/gcs_totals_by_country.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,243 @@
"Country","Count"
"Serbia and Montenegro","1189020000"
"United States","1001950000"
"Germany","12031900"
"United Kingdom","5348200"
"Brazil","1225402"
"France","776590"
"Spain","540230"
"Korea, Republic of","523913"
"Switzerland","518770"
"Canada","505720"
"Australia","501160"
"Netherlands","500610"
"Poland","467420"
"Indonesia","464766"
"Italy","337715"
"Japan","329490"
"Argentina","305668"
"Ireland","298550"
"Colombia","285080"
"Iran, Islamic Republic of","268490"
"Russian Federation","264007"
"India","252241"
"Belgium","236430"
"China","231860"
"Turkey","205764"
"Finland","186944"
"Sweden","176891"
"Mexico","174174"
"Denmark","171947"
"Slovenia","156250"
"Singapore","136608"
"South Africa","124099"
"Hong Kong","121683"
"Peru","98460"
"Norway","89557"
"Costa Rica","83621"
"Czech Republic","80608"
"Ecuador","78348"
"Ukraine","76967"
"Portugal","76889"
"Aruba","74692"
"Austria","73839"
"New Zealand","64452"
"Croatia (Hrvatska)","64051"
"Lithuania","52709"
"Greece","43780"
"Nicaragua","43722"
"Chile","43448"
"Malaysia","39233"
"Hungary","36620"
"Saudi Arabia","34136"
"Cuba","32785"
"Nigeria","30231"
"Romania","27824"
"Venezuela","26313"
"Israel","24304"
"Philippines","22414"
"Slovakia","21442"
"Bulgaria","18844"
"Pakistan","17099"
"Thailand","16742"
"Uzbekistan","15893"
"Luxembourg","15621"
"Taiwan, Province of China","15481"
"Iraq","15225"
"Qatar","12249"
"Moldova, Republic of","12082"
"Panama","11434"
"Iceland","11139"
"Egypt","9909"
"United Arab Emirates","9088"
"Ghana","8721"
"Somalia","8677"
"Azerbaijan","7437"
"Lebanon","7328"
"Kenya","7227"
"Armenia","6735"
"Vietnam","6483"
"El Salvador","5558"
"Bolivia","5198"
"Paraguay","5145"
"Algeria","5066"
"Nepal","4955"
"Latvia","4942"
"Cyprus","4922"
"Estonia","4781"
"Rwanda","3727"
"Virgin Islands, U.S.","3640"
"Madagascar","3566"
"Uganda","3555"
"Kazakhstan","3166"
"Sri Lanka","3143"
"Uruguay","2984"
"Congo, the Democratic Republic of the","2553"
"Bosnia and Herzegovina","2512"
"Ethiopia","2488"
"Malta","2465"
"Virgin Islands, British","2284"
"Jordan","2199"
"Tanzania, United Republic of","2130"
"Bangladesh","2017"
"Syrian Arab Republic","1926"
"Yemen","1758"
"Zimbabwe","1734"
"Libyan Arab Jamahiriya","1707"
"Macedonia, the Former Yugosalv Republic of","1699"
"Georgia","1541"
"Maldives","1426"
"Oman","1412"
"Saint Lucia","1392"
"Tonga","1381"
"Morocco","1202"
"Guatemala","1108"
"Dominican Republic","1071"
"Albania","1033"
"Bhutan","958"
"Tunisia","878"
"Cambodia","850"
"Namibia","816"
"Macao","682"
"Mozambique","600"
"Angola","553"
"Brunei Darussalam","544"
"Puerto Rico","456"
"Belarus","418"
"Palestinian Territory","411"
"Botswana","402"
"Malawi","388"
"Fiji","339"
"Sierra Leone","311"
"Zambia","310"
"Samoa","298"
"Burkina Faso","270"
"Faroe Islands","254"
"Afghanistan","252"
"Jamaica","249"
"Haiti","239"
"Kyrgyzstan","229"
"Bahrain","199"
"Myanmar","184"
"Trinidad and Tobago","168"
"Honduras","165"
"Tajikistan","149"
"New Caledonia","139"
"Mongolia","132"
"Liechtenstein","131"
"Mauritius","118"
"Benin","117"
"Senegal","112"
"Congo","112"
"Saint Helena","106"
"Lesotho","102"
"Saint Pierre and Miquelon","101"
"Sao Tome and Principe","97"
"Cameroon","91"
"Reunion","90"
"Holy See (Vatican City State)","75"
"Seychelles","70"
"Papua New Guinea","63"
"Cape Verde","59"
"Grenada","57"
"Chad","54"
"Kuwait","50"
"Greenland","47"
"Niger","43"
"Cote D'ivoire","42"
"Gambia","39"
"Barbados","36"
"Antarctica","36"
"Sudan","31"
"Lao People's Democratic Republic","27"
"Guyana","27"
"Belize","27"
"Monaco","26"
"Wallis and Futuna","24"
"Vanuatu","23"
"Togo","22"
"Christmas Island","21"
"Bermuda","21"
"Gibraltar","17"
"Saint Vincent and the Grenadines","16"
"San Marino","16"
"Antigua and Barbuda","15"
"Burundi","15"
"American Samoa","15"
"Andorra","14"
"Micronesia, Federated States of","13"
"Bahamas","13"
"Cayman Islands","13"
"Solomon Islands","13"
"Suriname","12"
"Norfolk Island","12"
"Mali","10"
"Guinea","9"
"Tuvalu","9"
"Eritrea","9"
"Niue","9"
"South Georgia and the South Sandwich Islands","8"
"Djibouti","8"
"Turkmenistan","7"
"Mauritania","7"
"Saint Kitts and Nevis","7"
"Falkland Islands (Malvinas)","6"
"Northern Mariana Islands","5"
"Swaziland","5"
"Nauru","5"
"Turks and Caicos Islands","5"
"Guinea-Bissau","5"
"Cook Islands","4"
"Equatorial Guinea","4"
"Palau","4"
"Anguilla","3"
"Liberia","3"
"Kiribati","2"
"Mayotte","1"
"Comoros","1"
"British Indian Ocean Territory","0"
"Western Sahara","0"
"Bouvet Island","0"
"Yugoslavia","0"
"United States Minor Outlying Islands","0"
"Netherlands Antilles","0"
"Tokelau","0"
"Central African Republic","0"
"Korea, Democratic People's Republic of","0"
"Heard Island and Mcdonald Islands","0"
"Guam","0"
"Guadeloupe","0"
"Pitcairn","0"
"Gabon","0"
"French Southern Territories","0"
"French Polynesia","0"
"French Guiana","0"
"France, Metropolitan","0"
"Martinique","0"
"Montserrat","0"
"European Union","0"
"East Timor","0"
"Dominica","0"
"Svalbard and Jan Mayen","0"
"Cocos (Keeling) Islands","0"
"Marshall Islands","0"
4 changes: 2 additions & 2 deletions data/2024Q4/2-process/gcs_totals_by_free_cultural.csv
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
"Category","Count"
"Approved for Free Cultural Works","26263961000"
"Limited uses","18561939832"
"Approved for Free Cultural Works","21006439000"
"Limited use","15762297041"
36 changes: 36 additions & 0 deletions data/2024Q4/2-process/gcs_totals_by_language.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
"Language","Count"
"English","933130000"
"Spanish","13907400"
"German","6040400"
"Portuguese","5156500"
"Indonesian","3293060"
"French","2127700"
"Italian","639900"
"Turkish","630040"
"Russian","628430"
"Polish","578500"
"Dutch","554300"
"Japanese","470320"
"Slovenian","376700"
"Chinese (Simplified)","356600"
"Korean","250660"
"Czech","236380"
"Swedish","180930"
"Serbian","162510"
"Romanian","159590"
"Croatian","155980"
"Catalan","151730"
"Norwegian","140300"
"Finnish","120640"
"Greek","110960"
"Hungarian","93330"
"Danish","72619"
"Arabic","70040"
"Chinese (Traditional)","69810"
"Lithuanian","61210"
"Slovak","58650"
"Latvian","46466"
"Hebrew","40703"
"Icelandic","27515"
"Bulgarian","24191"
"Estonian","20960"
8 changes: 0 additions & 8 deletions data/2024Q4/2-process/gcs_totals_by_product.csv

This file was deleted.

8 changes: 4 additions & 4 deletions data/2024Q4/2-process/gcs_totals_by_restrictions.csv
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"Category","Count"
"level 0","95700000"
"level 1","26168261000"
"level 2","7524932400"
"level 3","11037007432"
"level 0 - unrestricted","85680000"
"level 1 - few restrictions","20920759000"
"level 2 - some restrictions","5655765800"
"level 3 - many restrictions","10106531241"
19 changes: 0 additions & 19 deletions data/2024Q4/2-process/gcs_totals_by_unit.csv

This file was deleted.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added data/2024Q4/3-report/gcs_free_culture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added data/2024Q4/3-report/gcs_product_totals.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added data/2024Q4/3-report/gcs_tool_status.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading