Skip to content

Commit

Permalink
ORC-1299: Fix fetch data error in bench module
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?
This PR aims to fix bench error when fetch data.

### Why are the changes needed?
Fetch `https://s3.amazonaws.com/nyc-tlc/` will get a 403 error.

### How was this patch tested?
UT

Closes #1292 from deshanxiao/deshan/1299.

Authored-by: deshanxiao <deshanxiao@microsoft.com>
Signed-off-by: William Hyun <william@apache.org>
(cherry picked from commit 0638679)
Signed-off-by: William Hyun <william@apache.org>
  • Loading branch information
deshanxiao authored and williamhyun committed Oct 26, 2022
1 parent 38eb709 commit c03f010
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions java/bench/fetch-data.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,10 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -e
mkdir -p data/sources/taxi
(cd data/sources/taxi; wget https://s3.amazonaws.com/nyc-tlc/trip+data/yellow_tripdata_2015-11.parquet )
(cd data/sources/taxi; wget https://s3.amazonaws.com/nyc-tlc/trip+data/yellow_tripdata_2015-12.parquet )
(cd data/sources/taxi; wget https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2015-11.parquet )
(cd data/sources/taxi; wget https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2015-12.parquet )

mkdir -p data/sources/github
(cd data/sources/github; wget http://data.gharchive.org/2015-11-{01..15}-{0..23}.json.gz)

0 comments on commit c03f010

Please sign in to comment.