{"id":189,"date":"2020-09-02T17:00:05","date_gmt":"2020-09-02T11:30:05","guid":{"rendered":"https:\/\/ganofins.com\/blog\/?p=189"},"modified":"2020-09-02T18:05:51","modified_gmt":"2020-09-02T12:35:51","slug":"i-wrote-a-python-module-proxy-extractor","status":"publish","type":"post","link":"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/","title":{"rendered":"I wrote a Python module Proxy Extractor"},"content":{"rendered":"\n<p>Hello everyone,<br>In this post, I will tell you how I wrote a <a href=\"https:\/\/github.com\/Ganofins\/proxy-extractor\" target=\"_blank\" rel=\"noreferrer noopener\"><strong><span style=\"color:#0e00d3\" class=\"has-inline-color\">Python module + CLI<\/span><\/strong><\/a> script to fetch proxies from free proxy listing sites.<\/p>\n\n\n\n<p>I am a <em>part-time freelancer<\/em> and I get tons of projects related to data scraping such as scraping Instagram posts, scraping Udemy course details, scraping leads from LinkedIn, etc.<br>Now obviously, fetching contents from such sites in bulk in a short interval of time from the same IP will result in blocking your IP address.<br><br>Thus, in order to get contents from such sites without getting blocked, I use multiple solutions such as &#8211;<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><em>Using Proxies<\/em><\/li><li><em>Using random User-Agent with each request<\/em><\/li><li><em>Adding sleep interval between each request<\/em><\/li><li><em>Using referrer, authority, etc headers in the request<\/em><\/li><\/ul>\n\n\n\n<p>In this post, I will just talk about Proxies.<br>I use free proxies or paid proxies based upon the client&#8217;s demand and requirement.<br><br><strong>Paid proxies<\/strong> can be easily added in a list or array or file and can be used within the script.<br><br><em>But for using <\/em><strong>free proxies<\/strong><em> &#8211;<\/em><br>Firstly, I would need to find free proxy listing sites, then I would have to create a script to scrape proxies from there.<br>Then, I would have to create a function to check whether that proxy is working or not. Then, I will create a list or array to add only the working proxies into it.<br><br>Now, doing the above steps each time for each new script or project is very tiring and boring.<br>Thus, a few days ago I got an idea to write a Python module for that purpose which can also be used as Command Line Utility to fetch the free proxies.<\/p>\n\n\n\n<h3 class=\"has-text-align-center wp-block-heading\"><a href=\"https:\/\/github.com\/ganofins\/proxy-extractor\/\" target=\"_blank\" rel=\"noreferrer noopener\"><span style=\"color:#0e00d3\" class=\"has-inline-color\"><strong>Proxy Extractor<\/strong><\/span><\/a><\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1024\" height=\"432\" src=\"https:\/\/ganofins.com\/blog\/wp-content\/uploads\/2020\/09\/proxy-extractor-1024x432.png\" alt=\"\" class=\"wp-image-214\" srcset=\"https:\/\/ganofins.com\/blog\/wp-content\/uploads\/2020\/09\/proxy-extractor-1024x432.png 1024w, https:\/\/ganofins.com\/blog\/wp-content\/uploads\/2020\/09\/proxy-extractor-300x127.png 300w, https:\/\/ganofins.com\/blog\/wp-content\/uploads\/2020\/09\/proxy-extractor-768x324.png 768w, https:\/\/ganofins.com\/blog\/wp-content\/uploads\/2020\/09\/proxy-extractor.png 1289w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><figcaption>Proxy Extractor<\/figcaption><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<h4 class=\"wp-block-heading\"><br><br><strong>Now, let&#8217;s dive into its code and see how I wrote that module + CLI utility &#8211;<\/strong><br><br><\/h4>\n\n\n\n<p><span style=\"color:#067700\" class=\"has-inline-color\"><strong>1<\/strong>. <strong>proxy_extractor\/__init__.py<\/strong><\/span><br><br>Created this file to make Python treat this directory as a package.<br><br><br><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><strong><span style=\"color:#067700\" class=\"has-inline-color\">2. proxy_extractor\/useragents.py<\/span><\/strong><br><br>I created this file to store the list of User-Agents so that with each request a random user-agent can be used.<br>I wrote a function <strong><em>rand_user_agent<\/em><\/strong> to pick random user-agent from that list using the <a href=\"https:\/\/docs.python.org\/3\/library\/random.html\" target=\"_blank\" rel=\"noreferrer noopener nofollow\"><span style=\"color:#0e00d3\" class=\"has-inline-color\">random<\/span><\/a> module.<br><br><br><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><strong><span style=\"color:#067700\" class=\"has-inline-color\">3. proxy_extractor\/requester.py<\/span><\/strong><br><br>This file contains a function <strong><em>make_request<\/em><\/strong> to send an asynchronous request to the URL passed in that method.<br>To make an asynchronous request, I used <a href=\"https:\/\/docs.aiohttp.org\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\"><span style=\"color:#0e00d3\" class=\"has-inline-color\">aiohttp<\/span><\/a> module.<br>I also used function <strong><em>rand_user_agent<\/em><\/strong> from useragents.py to pick a random user-agent and pass with the request made by the <strong><em>make_request<\/em><\/strong> function.<br><br><br><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><strong><span style=\"color:#067700\" class=\"has-inline-color\">4. proxy_extractor\/scraper.py<\/span><\/strong><br><br>This file contains a function <strong><em>_extract_proxies_free_proxy_list_net<\/em><\/strong> to parse the HTML content requested from <a href=\"https:\/\/free-proxy-list.net\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\"><span style=\"color:#0e00d3\" class=\"has-inline-color\">free-proxy-list<\/span><\/a> with the help of <a href=\"https:\/\/pypi.org\/project\/beautifulsoup4\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\"><span style=\"color:#0e00d3\" class=\"has-inline-color\">bs4<\/span><\/a> module.<br><br><br><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><strong><span style=\"color:#067700\" class=\"has-inline-color\">5. proxy_extractor\/checker.py<\/span><\/strong><br><br>This file contains a async function <strong><em>proxy_check<\/em><\/strong> to check whether the passed proxy is working or not by aiohttp module and a random user-agent from the useragents.py with a timeout limit (default: 50 seconds).<br><br><br><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><em><strong><span class=\"has-inline-color has-vivid-purple-color\">**You can check out <\/span><a href=\"https:\/\/iq.opengenus.org\/asyncio-async-and-await-in-python\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\"><span class=\"has-inline-color has-vivid-red-color\">this post<\/span><\/a><span class=\"has-inline-color has-vivid-purple-color\"> in order to understand the basics about async, await, asyncio module.<\/span><\/strong><\/em><br><br><br><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><strong><span style=\"color:#067700\" class=\"has-inline-color\">6. proxy_extractor\/extractor.py<\/span><\/strong><br><br>This file contains several <a href=\"https:\/\/docs.python.org\/3\/library\/asyncio-task.html#coroutines\" target=\"_blank\" rel=\"noreferrer noopener nofollow\"><span style=\"color:#0e00d3\" class=\"has-inline-color\">coroutines<\/span><\/a> (async functions) &#8211;<br><br><br><strong><em><span style=\"color:#067700\" class=\"has-inline-color\">AsyncIter class(items) &#8211;<\/span><\/em><\/strong><br>This class takes a list as an input and returns an async list which can be used with async for loops.<br><br><br><strong><em><span style=\"color:#067700\" class=\"has-inline-color\">extract_proxies(https, proxy_count) &#8211; <\/span><\/em><\/strong><br>This async function fetches the HTML content from <a href=\"https:\/\/free-proxy-list.net\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\"><span style=\"color:#0e00d3\" class=\"has-inline-color\">free-proxy-list<\/span><\/a> using the <em>make_request<\/em> function of requester.py file and then that HTML content is parsed by the <em>_extract_proxies_free_proxy_list_net<\/em> function of scraper.py and it returns a parsed list consisting of IP addresses, their countries, their port, https support, or not, etc.<br><br><br><strong><em><span style=\"color:#067700\" class=\"has-inline-color\">get_extracted_proxies(proxy_list) &#8211;<\/span><\/em><\/strong><br>This function takes an async list and yields each element.<br><br><br><strong><em><span style=\"color:#067700\" class=\"has-inline-color\">proxy_remove(proxy, proxy_count, timeout) &#8211;<\/span><\/em><\/strong><br>This function checks the passed proxy is working or not and only adds a working proxy to a <em>working_proxy_list<\/em> list.<br><br><br><strong><em><span style=\"color:#067700\" class=\"has-inline-color\">create_jobs(proxy_list, https, proxy_count, timeout) &#8211;<\/span><\/em><\/strong><br>This function adds each proxy element yielded from the <em>get_extracted_proxies <\/em>function to a tasks list.<br><br><br><strong><em><span style=\"color:#067700\" class=\"has-inline-color\">extract_proxy(https=False, proxy_count=100, timeout=50) &#8211;<\/span><\/em><\/strong><br>This function firstly extracts the proxies using the <em>extract_proxies<\/em> function and then creates a <em>new_event_loop<\/em> with the use of <a href=\"https:\/\/docs.python.org\/3\/library\/asyncio.html\" target=\"_blank\" rel=\"noreferrer noopener nofollow\"><span style=\"color:#0e00d3\" class=\"has-inline-color\">asyncio<\/span><\/a> module.<br><em>*An event loop of asyncio runs asynchronous tasks and callbacks.<\/em><br>Then, I used the <em>set_event_loop<\/em> method of asyncio to set the newly created loop as the current loop.<br>Then, that extracted proxy list is passed to the <em>AsyncIter<\/em> class to get an async list of proxies. After that, I used <em>run_until_complete<\/em> method of asyncio to run the <em>create_jobs<\/em> function until its completion.<br>At last, it returns a list of working proxies.<br><br><br><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><strong><span style=\"color:#067700\" class=\"has-inline-color\">7. color.py<\/span><\/strong><br><br>I used this file to assign the color code values based on the operating system a user is using.<br><br><br><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><strong><span style=\"color:#067700\" class=\"has-inline-color\">8. cli.py<\/span><\/strong><br><br>Using this file, one can use this Python module as a Command-Line utility. I have used <a href=\"https:\/\/docs.python.org\/3\/library\/argparse.html\" target=\"_blank\" rel=\"noreferrer noopener nofollow\"><span style=\"color:#0e00d3\" class=\"has-inline-color\">argparse<\/span><\/a> module for parsing the command arguments. It calls the <em>extract_proxy<\/em> function of <em>proxy_extractor\/extractor.py<\/em> file.<br><br><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p>Those were the code files used to build this Python module + CLI utility. You can check it out <a href=\"https:\/\/github.com\/Ganofins\/proxy-extractor\" target=\"_blank\" rel=\"noreferrer noopener\"><span style=\"color:#0e00d3\" class=\"has-inline-color\">here<\/span><\/a> and can also read about its installation and usage.<\/p>\n\n\n\n<p><strong>Feel free to contribute to this project.<\/strong><br>I hope you got some insights into my module.<br><\/p>\n\n\n\n<p><strong>Want to hire me?<\/strong><br><a href=\"mailto:bagariaganesh@gmail.com\"><span style=\"color:#0e00d3\" class=\"has-inline-color\">Email<\/span><\/a><span style=\"color:#0e00d3\" class=\"has-inline-color\"><br><\/span><a href=\"https:\/\/fiverr.com\/ganofins\" target=\"_blank\" rel=\"noreferrer noopener nofollow\"><span style=\"color:#0e00d3\" class=\"has-inline-color\">Fiverr<\/span><\/a><\/p>\n\n\n\n<p>Got a question or suggestion? Comment below.<br><strong>Thank you so much for reading this post.<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hello everyone,In this post, I will tell you how I wrote a Python module + CLI script to fetch proxies from free proxy listing sites. I am a part-time freelancer and I get tons of projects related to data scraping&#8230;<\/p>\n","protected":false},"author":1,"featured_media":214,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[38,37],"tags":[46,13,49,41,45,44,42,48,43,47,12],"class_list":["post-189","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-language","category-python","tag-free-proxies","tag-ganofins","tag-ganofins-proxy-extractor","tag-proxy-extractor","tag-proxy-extractor-cli","tag-proxy-extractor-ganofins","tag-proxy-extractor-module","tag-proxy-extractor-script","tag-python-proxy-extractor","tag-python-proxy-module","tag-tutorials-with-ganofins"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.8 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>I wrote a Python module Proxy Extractor - Ganofins Blog<\/title>\n<meta name=\"description\" content=\"Proxy Extractor is a Python module + CLI utility to extract free proxies from various proxy listing sites created by Ganofins\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"I wrote a Python module Proxy Extractor - Ganofins Blog\" \/>\n<meta property=\"og:description\" content=\"Proxy Extractor is a Python module + CLI utility to extract free proxies from various proxy listing sites created by Ganofins\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/\" \/>\n<meta property=\"og:site_name\" content=\"Ganofins Blog\" \/>\n<meta property=\"article:published_time\" content=\"2020-09-02T11:30:05+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-09-02T12:35:51+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/ganofins.com\/blog\/wp-content\/uploads\/2020\/09\/proxy-extractor.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1289\" \/>\n\t<meta property=\"og:image:height\" content=\"544\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"ganofins\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"ganofins\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/\"},\"author\":{\"name\":\"ganofins\",\"@id\":\"https:\/\/ganofins.com\/blog\/#\/schema\/person\/ce75d64682bcbf85acac97c9cb93a8db\"},\"headline\":\"I wrote a Python module Proxy Extractor\",\"datePublished\":\"2020-09-02T11:30:05+00:00\",\"dateModified\":\"2020-09-02T12:35:51+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/\"},\"wordCount\":905,\"commentCount\":6,\"publisher\":{\"@id\":\"https:\/\/ganofins.com\/blog\/#\/schema\/person\/ce75d64682bcbf85acac97c9cb93a8db\"},\"image\":{\"@id\":\"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/ganofins.com\/blog\/wp-content\/uploads\/2020\/09\/proxy-extractor.png\",\"keywords\":[\"free proxies\",\"ganofins\",\"ganofins proxy-extractor\",\"proxy extractor\",\"proxy extractor cli\",\"proxy extractor ganofins\",\"proxy extractor module\",\"proxy extractor script\",\"python proxy extractor\",\"python proxy module\",\"tutorials with ganofins\"],\"articleSection\":[\"Language\",\"Python\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/\",\"url\":\"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/\",\"name\":\"I wrote a Python module Proxy Extractor - Ganofins Blog\",\"isPartOf\":{\"@id\":\"https:\/\/ganofins.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/ganofins.com\/blog\/wp-content\/uploads\/2020\/09\/proxy-extractor.png\",\"datePublished\":\"2020-09-02T11:30:05+00:00\",\"dateModified\":\"2020-09-02T12:35:51+00:00\",\"description\":\"Proxy Extractor is a Python module + CLI utility to extract free proxies from various proxy listing sites created by Ganofins\",\"breadcrumb\":{\"@id\":\"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/#primaryimage\",\"url\":\"https:\/\/ganofins.com\/blog\/wp-content\/uploads\/2020\/09\/proxy-extractor.png\",\"contentUrl\":\"https:\/\/ganofins.com\/blog\/wp-content\/uploads\/2020\/09\/proxy-extractor.png\",\"width\":1289,\"height\":544,\"caption\":\"Proxy Extractor\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/ganofins.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"I wrote a Python module Proxy Extractor\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/ganofins.com\/blog\/#website\",\"url\":\"https:\/\/ganofins.com\/blog\/\",\"name\":\"Ganofins Blog\",\"description\":\"Learn Today Apply Today\",\"publisher\":{\"@id\":\"https:\/\/ganofins.com\/blog\/#\/schema\/person\/ce75d64682bcbf85acac97c9cb93a8db\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/ganofins.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/ganofins.com\/blog\/#\/schema\/person\/ce75d64682bcbf85acac97c9cb93a8db\",\"name\":\"ganofins\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/ganofins.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/ganofins.com\/blog\/wp-content\/litespeed\/avatar\/a1fd68d9e27b63b4007d850d5d958b59.jpg?ver=1776081109\",\"contentUrl\":\"https:\/\/ganofins.com\/blog\/wp-content\/litespeed\/avatar\/a1fd68d9e27b63b4007d850d5d958b59.jpg?ver=1776081109\",\"caption\":\"ganofins\"},\"logo\":{\"@id\":\"https:\/\/ganofins.com\/blog\/#\/schema\/person\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"I wrote a Python module Proxy Extractor - Ganofins Blog","description":"Proxy Extractor is a Python module + CLI utility to extract free proxies from various proxy listing sites created by Ganofins","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/","og_locale":"en_US","og_type":"article","og_title":"I wrote a Python module Proxy Extractor - Ganofins Blog","og_description":"Proxy Extractor is a Python module + CLI utility to extract free proxies from various proxy listing sites created by Ganofins","og_url":"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/","og_site_name":"Ganofins Blog","article_published_time":"2020-09-02T11:30:05+00:00","article_modified_time":"2020-09-02T12:35:51+00:00","og_image":[{"width":1289,"height":544,"url":"https:\/\/ganofins.com\/blog\/wp-content\/uploads\/2020\/09\/proxy-extractor.png","type":"image\/png"}],"author":"ganofins","twitter_card":"summary_large_image","twitter_misc":{"Written by":"ganofins","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/#article","isPartOf":{"@id":"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/"},"author":{"name":"ganofins","@id":"https:\/\/ganofins.com\/blog\/#\/schema\/person\/ce75d64682bcbf85acac97c9cb93a8db"},"headline":"I wrote a Python module Proxy Extractor","datePublished":"2020-09-02T11:30:05+00:00","dateModified":"2020-09-02T12:35:51+00:00","mainEntityOfPage":{"@id":"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/"},"wordCount":905,"commentCount":6,"publisher":{"@id":"https:\/\/ganofins.com\/blog\/#\/schema\/person\/ce75d64682bcbf85acac97c9cb93a8db"},"image":{"@id":"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/#primaryimage"},"thumbnailUrl":"https:\/\/ganofins.com\/blog\/wp-content\/uploads\/2020\/09\/proxy-extractor.png","keywords":["free proxies","ganofins","ganofins proxy-extractor","proxy extractor","proxy extractor cli","proxy extractor ganofins","proxy extractor module","proxy extractor script","python proxy extractor","python proxy module","tutorials with ganofins"],"articleSection":["Language","Python"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/","url":"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/","name":"I wrote a Python module Proxy Extractor - Ganofins Blog","isPartOf":{"@id":"https:\/\/ganofins.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/#primaryimage"},"image":{"@id":"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/#primaryimage"},"thumbnailUrl":"https:\/\/ganofins.com\/blog\/wp-content\/uploads\/2020\/09\/proxy-extractor.png","datePublished":"2020-09-02T11:30:05+00:00","dateModified":"2020-09-02T12:35:51+00:00","description":"Proxy Extractor is a Python module + CLI utility to extract free proxies from various proxy listing sites created by Ganofins","breadcrumb":{"@id":"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/#primaryimage","url":"https:\/\/ganofins.com\/blog\/wp-content\/uploads\/2020\/09\/proxy-extractor.png","contentUrl":"https:\/\/ganofins.com\/blog\/wp-content\/uploads\/2020\/09\/proxy-extractor.png","width":1289,"height":544,"caption":"Proxy Extractor"},{"@type":"BreadcrumbList","@id":"https:\/\/ganofins.com\/blog\/i-wrote-a-python-module-proxy-extractor\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ganofins.com\/blog\/"},{"@type":"ListItem","position":2,"name":"I wrote a Python module Proxy Extractor"}]},{"@type":"WebSite","@id":"https:\/\/ganofins.com\/blog\/#website","url":"https:\/\/ganofins.com\/blog\/","name":"Ganofins Blog","description":"Learn Today Apply Today","publisher":{"@id":"https:\/\/ganofins.com\/blog\/#\/schema\/person\/ce75d64682bcbf85acac97c9cb93a8db"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ganofins.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/ganofins.com\/blog\/#\/schema\/person\/ce75d64682bcbf85acac97c9cb93a8db","name":"ganofins","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ganofins.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/ganofins.com\/blog\/wp-content\/litespeed\/avatar\/a1fd68d9e27b63b4007d850d5d958b59.jpg?ver=1776081109","contentUrl":"https:\/\/ganofins.com\/blog\/wp-content\/litespeed\/avatar\/a1fd68d9e27b63b4007d850d5d958b59.jpg?ver=1776081109","caption":"ganofins"},"logo":{"@id":"https:\/\/ganofins.com\/blog\/#\/schema\/person\/image\/"}}]}},"amp_enabled":true,"post_mailing_queue_ids":[],"_links":{"self":[{"href":"https:\/\/ganofins.com\/blog\/wp-json\/wp\/v2\/posts\/189","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ganofins.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ganofins.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ganofins.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ganofins.com\/blog\/wp-json\/wp\/v2\/comments?post=189"}],"version-history":[{"count":52,"href":"https:\/\/ganofins.com\/blog\/wp-json\/wp\/v2\/posts\/189\/revisions"}],"predecessor-version":[{"id":265,"href":"https:\/\/ganofins.com\/blog\/wp-json\/wp\/v2\/posts\/189\/revisions\/265"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ganofins.com\/blog\/wp-json\/wp\/v2\/media\/214"}],"wp:attachment":[{"href":"https:\/\/ganofins.com\/blog\/wp-json\/wp\/v2\/media?parent=189"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ganofins.com\/blog\/wp-json\/wp\/v2\/categories?post=189"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ganofins.com\/blog\/wp-json\/wp\/v2\/tags?post=189"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}