Skip to content

Releases: m-lab/etl

Update NDT5 & 7 descriptions for parser efficiency

23 Jul 12:02
4daf8ba
Compare
Choose a tag to compare
Schema docs update2 (#916)

* updated schema descriptions so they can be used across different parts of the record-- for example - common descriptions applied to S2C and C2S.tcpinfo

* Update bindata

* test removal of EOL whitespace

* Update bindata

Update NDT7ResultRow.yaml

21 Jul 20:15
b485d00
Compare
Choose a tag to compare

Updates the schema descriptions for ndt7

Latest generate-schema-docs

13 Jul 22:29
b985084
Compare
Choose a tag to compare
v2.4.0

Build and ship in same environment (#903)

v2.2.4: Add sink.close (#898)

13 Jul 20:15
aacc0b3
Compare
Choose a tag to compare
* add json annotations

* improve error logging

* add Sink.Close

* travis log each outcome

* improve comments on errors

* return Put errors

* disable manual GCS test

* improve ProcessAll error handling comments

* Add closer to task instead of parser

* move row.NullCloser

* update bindata

Migrate to go/bqx, NDT7 Standard Columns, use BytesAcked

21 May 17:06
03127d7
Compare
Choose a tag to compare

Many changes, mostly for universal K8S.
Migrates to go/bqx to avoid deprecated packages.

Add BIGQUERY_DATASET for GKE deploymment (#852) … e2397de
Add NDT7 parser using Standard Columns for NDT7ResultRow (#854) … 50ff007
ndt7: Use BytesAcked to calculate MeanThroughputMbps (#856) 52c6638
Stream direct to partitioned table for GKE parsers (#853) … 59d98ea
Add parser for annotation datatype (#858) … 9495a52
Allow json.gz files for ndt7 (#861) … 2df1d2f
Fix IsParsable again (#862) … 56512b9
Use fakestorage, extract sink type from BQInserter (#859) … 8f6289a
Fix bad injection (#864) … bf39cae
Add factories for injection (#865) … a6d2ff5
Reorganize factory code to avoid import cycle (#869) … 6214ded
inject task.Factory into ProcessGKETask (#870) … fa1bca4
migrate from go/bqx to go/cloud/bqx (#871) … 03127d7

BAD RELEASE Duplicate release to force new rollout

21 May 16:17
caa2a5a
Compare
Choose a tag to compare

This is just a new release of the existing March17 commit. GCE Enforcer changed firewall rules. This is an attempt to force new rollout of all existing services.

This actually failed, because deprecated bqx package is no longer available from m-lab/go.

Many refactorings.

17 Mar 22:46
caa2a5a
Compare
Choose a tag to compare

This release is mostly a bunch of refactorings. There are some small improvements in log messages.
It also incorporates changes to the Gardener job/ service, important only to the GKE prototype parsers.

Refactor ETLSource (#845) …
Tweak previous refactoring. (#847) …
row.Stats improvements (#848) …
Migrate ndt_result to use row.Base. (#849) …
delete spammy debug logs (#851) …
Use new NDT5Result struct throughout (#850) …

tcpinfo parser, log improvement and etc.

11 Mar 15:49
8d72fb9
Compare
Choose a tag to compare

#834 Add row package with Sink interface

#835 Implement tcpinfo parser using row.Base

#836 Add simpler "Sink" api

#837 Add TestCount stats

#840 Add GCS backed rowwriter

#841 use local job.NextJob for transition

#843 better insert error log

#844 Fix goroutine "leak" introduced last week in tcpinfo

traceroute annotation bugfix, 10% tcpinfo snapshot, better insert logs

27 Feb 14:16
9a7736e
Compare
Choose a tag to compare

NOTE: The current production code is experiencing about 8k backend failures per hour, most likely tcpinfo quota exceeded.

The following PRs have been submitted since 2.2.3

better insert error log (#828) …
keep only 1/10 of snapshots (#832) …
Fix the missing last few tests for a tarball (#830) …

There was also #825, but it was reverted.
Update unit tests to use changed Geo field names (#825) …
Revert "Update unit tests to use changed Geo field names (#825)" (#826) …

Generate Rich MD text when possible

12 Feb 20:10
18a6d54
Compare
Choose a tag to compare

NOTE: vA.B.C does not trigger app engine release to production. Use prod-vA.B.C

Generate rich md text when Discussion & Kernel keys are found (#816) …
Automate the schema change in travis (#815) …