11# proxy_py
22
3- proxy_py is a program which collects proxies, saves them in database and makes periodically checks. It has server for getting proxies with nice API(see below).
3+ proxy_py is a program which collects proxies, saves them in a database
4+ and makes periodically checks.
5+ It has a server for getting proxies with nice API(see below).
46
57## How to build?
68
@@ -19,7 +21,7 @@ pip3 install -r requirements.txt
1921
2022` cp config_examples/settings.py proxy_py/settings.py `
2123
22- 4 (Optional) Change database in settings.py file
24+ 4 Install postgresql and change database configuration in settings.py file
2325
24265 (Optional) Configure alembic
2527
@@ -31,13 +33,18 @@ pip3 install -r requirements.txt
3133
3234## I'm too lazy. Can I just use it?
3335
34- Yes, you can download virtualbox image [ here] ( https://drive.google.com/file/d/1oPf6xwOADRH95oZW0vkPr1Uu_iLDe9jc/view?usp=sharing ) . After downloading check that port forwarding is still working, you need forwarding of 55555 host port to 55555 guest.
36+ Yes, you can download virtualbox image
37+ [ here] ( https://drive.google.com/file/d/1oPf6xwOADRH95oZW0vkPr1Uu_iLDe9jc/view?usp=sharing ) .
38+ After downloading check that port forwarding is still working,
39+ you need forwarding of 55555 host port to 55555 guest.
3540
3641## How to get proxies?
3742
38- proxy_py has server based on aiohttp which is listening 127.0.0.1:55555
39- (you can change it in settings file) and provides proxies.
40- To get proxies you should send following json request:
43+ proxy_py has a server, based on aiohttp, which is listening 127.0.0.1:55555
44+ (you can change it in the settings file) and provides proxies.
45+ To get proxies you should send the following json request
46+ on address ` http://127.0.0.1:55555/api/v1/ `
47+ (or other domain if behind reverse proxy):
4148
4249``` json
4350{
@@ -47,10 +54,11 @@ To get proxies you should send following json request:
4754}
4855```
4956
50- Note: order_by makes result sorting by one or more fields separated by comma.
57+ Note: order_by makes the result sorted
58+ by one or more fields(separated by comma).
5159You can skip it. The required fields are ` model ` and ` method ` .
5260
53- It will return json response like this:
61+ It's gonna return you the json response like this:
5462
5563``` json
5664{
@@ -75,7 +83,7 @@ It will return json response like this:
7583```
7684
7785Note: All fields except * protocol* , * domain* , * port* , * auth_data* ,
78- * checking_period* and * address* can be null
86+ * checking_period* and * address* CAN be null
7987
8088Or error if something went wrong:
8189
@@ -91,13 +99,13 @@ Note: status_code is also duplicated in HTTP status code
9199
92100Example using curl:
93101
94- ` curl -X POST http://example.com :55555 -H "Content-Type: application/json" --data '{"model": "proxy", "method": "get"}' `
102+ ` curl -X POST http://127.0.0.1 :55555/api/v1/ -H "Content-Type: application/json" --data '{"model": "proxy", "method": "get"}' `
95103
96104Example using httpie:
97105
98- ` http POST http://example.com :55555 model=proxy method=get `
106+ ` http POST http://127.0.0.1 :55555/api/v1/ model=proxy method=get `
99107
100- Example using python requests library:
108+ Example using python's ` requests ` library:
101109
102110``` python
103111import requests
@@ -111,7 +119,7 @@ def get_proxies():
111119 " method" : " get" ,
112120 }
113121
114- response = requests.post(' http://example.com :55555' , json = json_data)
122+ response = requests.post(' http://127.0.0.1 :55555/api/v1/ ' , json = json_data)
115123 if response.status_code == 200 :
116124 response = json.loads(response.text)
117125 for proxy in response[' data' ]:
@@ -136,7 +144,7 @@ async def get_proxies():
136144 }
137145
138146 async with aiohttp.ClientSession() as session:
139- async with session.post(' http://example.com :55555' , json = json_data) as response:
147+ async with session.post(' http://127.0.0.1 :55555/api/v1/ ' , json = json_data) as response:
140148 if response.status == 200 :
141149 response = json.loads(await response.text())
142150 for proxy in response[' data' ]:
@@ -152,9 +160,13 @@ async def get_proxies():
152160
153161Read more about API [ here] ( https://github.com/DevAlone/proxy_py/tree/master/docs/API.md )
154162
163+ ## How to contribute?
164+
165+ ` TODO: write guide about it `
166+
155167## How to test it?
156168
157- If you made changes to code and want to check that you didn't break
169+ If you made the changes to code and want to check that you didn't break
158170anything, go [ here] ( https://github.com/DevAlone/proxy_py/tree/master/docs/tests.md )
159171
160172## How to deploy on production using supervisor, nginx and postgresql in 8 steps?
@@ -165,20 +177,25 @@ anything, go [here](https://github.com/DevAlone/proxy_py/tree/master/docs/tests.
165177
1661782 Create virtual environment and install requirements on it
167179
168- 3 Install psycopg2
180+ 3 Copy settings.py example:
169181
170- ` ( proxy_py) proxy_py @server:~/proxy_py$ pip3 install psycopg2 `
182+ ` proxy_py@server:~/proxy_py$ cp config_examples/settings.py proxy_py/ `
171183
172- 4 create unprivileged user in postgresql database and add database authentication data to settings.py
184+ 4 create unprivileged user in postgresql database
185+ and change database authentication data in settings.py
173186
174187``` bash
175188proxy_py@server:~ /proxy_py$ vim proxy_py/settings.py
176189```
177190
178191``` bash
179- DATABASE_CONNECTION_ARGS = (
180- ' postgresql://USERNAME:PASSWORD@localhost/DB_NAME' ,
181- )
192+ DATABASE_CONNECTION_KWARGS = {
193+ ' database' : ' YOUR_POSTGRES_DATABASE' ,
194+ ' user' : ' YOUR_POSTGRES_USER' ,
195+ ' password' : ' YOUR_POSTGRES_PASSWORD' ,
196+ # number of simultaneous connections
197+ # 'max_connections': 20,
198+ }
182199```
183200
1842015 Copy supervisor config example and change it for your case
@@ -196,7 +213,7 @@ root@server:~$ ln -s /etc/nginx/sites-available/proxy_py /etc/nginx/sites-enable
196213root@server:~ $ vim /etc/nginx/sites-available/proxy_py
197214```
198215
199- 7 Restart supervisor and nginx
216+ 7 Restart supervisor and Nginx
200217
201218``` bash
202219root@server:~ $ supervisorctl reread
@@ -209,4 +226,4 @@ root@server:~$ /etc/init.d/nginx restart
209226
210227## What is it depend on?
211228
212- See requirements.txt
229+ See ` requirements.txt `
0 commit comments