Skip to content

Commit

Permalink
v0.9.0 (#97)
Browse files Browse the repository at this point in the history
* chore(cargo): update redis-macros requirement from 0.2.1 to 0.3.0

Updates the requirements on [redis-macros](https://github.com/daniel7grant/redis-macros) to permit the latest version.
- [Release notes](https://github.com/daniel7grant/redis-macros/releases)
- [Changelog](https://github.com/daniel7grant/redis-macros/blob/master/Changelog.md)
- [Commits](daniel7grant/redis-macros@v0.2.1...v0.3.0)

---
updated-dependencies:
- dependency-name: redis-macros
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* ci(actions): bump actions/checkout from 2 to 4

Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v2...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix `src/model/item.rs` (#92)

* Moved vinted-db-feeder as submodule

* Fix/advanced item test (#98)

* Updated item JSON

* Upgraded testing quality

* Storing test results in /output

* Added more new JSON fields

* Using lazy_static for logging

* Updated namespace for debug

* Removed lazy initialization

* Fixed clippy

* Added new SerdeError

* fixed clippy

* Bump v0.9.0

* chore(cargo): update redis requirement from 0.24.0 to 0.25.4 (#93)

Updates the requirements on [redis](https://github.com/redis-rs/redis-rs) to permit the latest version.
- [Release notes](https://github.com/redis-rs/redis-rs/releases)
- [Commits](redis-rs/redis-rs@redis-0.24.0...redis-0.25.4)

---
updated-dependencies:
- dependency-name: redis
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Álvaro Cabo Ciudad <alvarocaboac2@gmail.com>

* chore(cargo): update typed-builder requirement from 0.18 to 0.19 (#96)

---
updated-dependencies:
- dependency-name: typed-builder
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Álvaro Cabo Ciudad <alvarocaboac2@gmail.com>

* Updated CHANGELOG

* chore(cargo): update reqwest_cookie_store requirement from 0.6 to 0.8 (#94)

* chore(cargo): update reqwest_cookie_store requirement from 0.6 to 0.8

Updates the requirements on [reqwest_cookie_store](https://github.com/pfernie/reqwest_cookie_store) to permit the latest version.
- [Changelog](https://github.com/pfernie/reqwest_cookie_store/blob/main/CHANGELOG.md)
- [Commits](pfernie/reqwest_cookie_store@v0.6.0...v0.8.0)

---
updated-dependencies:
- dependency-name: reqwest_cookie_store
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix reqwest version

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Álvaro Cabo Ciudad <alvarocaboac2@gmail.com>

* fix method naming

* Removed unnecessary @staticmethod tag

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Baptiste Zahnow <75901978+LelouchFR@users.noreply.github.com>
  • Loading branch information
3 people authored Jul 28, 2024
1 parent ef8c116 commit 8292951
Show file tree
Hide file tree
Showing 68 changed files with 105,376 additions and 44 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/cargo_publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
name: Publish to crates registry
steps:
- name: Checkout sources
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: Install stable toolchain
uses: actions-rs/toolchain@v1
with:
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,5 @@ Cargo.lock
.vscode/
/**/results/
docker/query.sh

src/tests/output
4 changes: 0 additions & 4 deletions .gitmodules

This file was deleted.

16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,21 @@
# Changelog

# 0.9.0 (2024-07-28) [#97](https://github.com/TuTarea/vinted-rs/pull/97/)

## Fixed

- Updated API fields for `advancedItem`

## Improved

- Moved the [scrapping scripts](https://github.com/ThalosES/vinted-db-feeder) in `/scrapping` folder and removed the old repo

- update `reqwest_cookie_store` requirement from 0.6 to `0.8` [#94](https://github.com/ThalosES/vinted-rs/pull/94)

- update `typed-builder` requirement from 0.18 to `0.19` [#96](https://github.com/ThalosES/vinted-rs/pull/96)

- update `redis` requirement from 0.24.0 to `0.25.4` [#93](https://github.com/ThalosES/vinted-rs/pull/93)

# 0.8.7 (2024-04-3) [#82](https://github.com/TuTarea/vinted-rs/pull/82/)

## Improved
Expand Down
18 changes: 10 additions & 8 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "vinted-rs"
version = "0.8.7"
version = "0.9.0"
edition = "2021"
repository = "https://github.com/TuTarea/vinted-rs"
authors = [
Expand All @@ -25,22 +25,24 @@ minimal = []


[dependencies]
reqwest = { version = "0.11", features = ["json", "cookies"] }
reqwest = { version = "0.12", features = ["json", "cookies"] }
tokio = { version = "1", features = ["full"] }
serde = { version = "1.0", features = ["serde_derive"] }
thiserror = "1.0"
rand = "0.8"
reqwest_cookie_store = "0.6"
typed-builder = "0.18"
reqwest_cookie_store = "0.8"
typed-builder = "0.19"
fang = { version = "0.10.3", features = ["asynk"], default-features = false }
redis-macros = { version = "0.2.1", optional = true }
redis = { version = "0.25.3", optional = true }
redis-macros = { version = "0.3.0", optional = true }
redis = { version = "0.25.4", optional = true }
serde_json = { version = "1.0.91" }
log = "0.4.20"
lazy_static = "1.4.0"
[dev-dependencies]
redis-macros = { version = "0.2.1" }
redis = { version = "0.25.3" }
env_logger = "0.11.5"
redis-macros = { version = "0.3.0" }
redis = { version = "0.25.4" }


[dependencies.bb8-postgres]
version = "0.8"
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Via `cargo` you can add the library to your project's `Cargo.toml`

```toml
[dependencies]
vinted-rs = "0.8.7"
vinted-rs = "0.9.0"
```

## DB setup
Expand Down
5 changes: 5 additions & 0 deletions scrapping/vinted-db-feeder/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
data/*
!data/raw

pyvenv
*/__pycache__
76 changes: 76 additions & 0 deletions scrapping/vinted-db-feeder/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Vinted data scrapper

A series of data scrappers in Python that extract static information from the Vinted source code, being the perfect complement for a Vinted API wrapper.

## Install & Run

### Requirements

- Pip
- Python

1. Create a virtual environment

```bash
python3 -m venv pyvenv/
source venv/bin/activate
```

2. Run `requirements.txt`

```bash
pip3 install -r requirements.txt
```

3. Run `main.py`

## Extracted data categories

| Element | Fields Returned |
| ------------------- | --------------------------------------------------- |
| Brands | Names, Ids |
| Materials | Id, Name |
| Colors | Id, Color, Hex Code |
| Sizes | Id, Title, Size_Type, Category_id |
| Categories | Id, Title, Code, Parent Id, URL, URL EN, Item Count |
| Categories Children | Category Id, Child Id |
| Countries | Id, French_name, local_name, ISO_code, flag_emoji |

### Materials and sizes

- Available languages: 🇪🇸 🇫🇷 🇺🇸
- [More languages can be added if html file included in `data/raw/materials` or `data/raw/sizes`]

### Categories - Catalogs

- **Debug mode:** Builds the full decision tree

- **Exec mode:** Returns 2 CSVs:

- `categories.csv`: Table of all the available categories and their attributes

- `categories_children.csv`: Dictionary that models the Category->Children list relationship

## Performance

- Without brands search:

```bash
real 0m1,941s
user 0m1,225s
sys 0m0,037s
```

- Brands validation process:

```bash
real 14m14,211s
user 0m19,753s
sys 0m1,229s
```

## Authors

[Álvaro Cabo](https://github.com/alvarocabo)

[Pepe Márquez](https://github.com/pxp9)
1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/A_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/B_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/C_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/D_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/E_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/F_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/G_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/H_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/I_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/J_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/K_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/L_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/M_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/N_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/O_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/P_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/R_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/S_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/T_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/U_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/V_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/W_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/Y_elements.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions scrapping/vinted-db-feeder/data/raw/brands/Z_elements.html

Large diffs are not rendered by default.

Loading

0 comments on commit 8292951

Please sign in to comment.