github-featured / apify--crawlee

发布

0 发布 649 Git 标签

RSS 订阅源
  • v0.21.0-beta.4 1f7bf2392e
    比较

    Proxy configuration (#681)

    frostbyte_neo 发布于 2020-05-17 09:10:39 +00:00

    • Made proxy configuration class

    • Added createProxyConfiguration function to apify module

    • Added proxy configuration to basic crawler

    • Added proxy configuration to cheerio crawler

    • Proxy configuration class changes

    • Added private (fix)

    • Updated cheerio crawler tests for apify proxy

    • Added tests for proxy configuration class

    • Added proxy configuration to docs sidebars

    • test fixed, added proxy configuration class to apify module

    • fixed log

    • temporary removed

    • proxy object renamed to proxyInfo according to the @typedef

    • Removed section for proxy configuration, changed naming

    • Naming changes - proxyInfo object, added test for checking session identifier is correctly passed

    • ProxyConfiguration class option removed from Apify module

    • proxy.js file renamed to proxy_configuration.js, added code examples, Apify UI input configuration option, validation changes

    • Added warning log and actor UI input schema option tests

    • Added proxyInfo object to typedef

    • Fixed proxyConfiguration test

    • Fixed proxyConfiguration tests

    • Fixed cheerio crawler handlePageFunction proxy configuration tests

    • Description changes

    • Added proxyInfo object to handlePageFunction context

    • minor changes

    • Description changes connected to the proxyConfiugration and cheerioCrawler

    • Feature/proxy configuration puppeteer (#674)

    • Added proxyConfiguration to puppeteer crawler

    • Added tests for proxyConfiguration

    • Added proxyConfiguration to puppeteer, test changes

    • Added test for proxyConfiguration

    • Added proxyConfiguration option to requestAsBrowser

    • Changed method to regular expression

    • launchPuppeteer expects now only proxyUrl

    • PuppeteerPool now creates proxyUrl by provided proxyConfiguration

    • requestAsBrowser accept only the proxyUrl

    • Description added

    • minor changes

    • PuppeteerCrawler passes provided proxyConfiguration to PuppeteerPool

    • Fixed proxyConfiguration test

    • Fixed desc

    • Fixed test

    • Description changes

    • Description changes & removed hostname and port from proxyConfigurationOptions

    • Updated proxyConfiguration tests for cheerioCrawler and puppeteerCrawler

    • Updated puppeteerPool example usage

    • PuppeteerPool and puppeteer tests changes

    • Useless import removed

    • Removed getApifyProxyUrl method

    • Updated session management guide with proxy configuration

    • Added session validation to getInfo() and getUrl() methods

    • removed useless imports

    • Fix docs

    Co-authored-by: Ondra Urban ondra@apify.com

    • Add ProxyInfo type to BasicCrawler

    • Remove auto-added def

    • Add ProxyInfo types to PuppeteerCrawler

    • Remove redundant type

    • Improve ProxyInfo desc

    • Improve ProxyConfiguration desc

    • Remove hostname and port from ProxyConf constructor

    • Remove hostname and port from ProxyConfiguration constructor

    • Improve Apify.createProxyConf desc

    • Improve PuppeteerPool comments

    • Improve test

    Co-authored-by: Jan Bárta jbarta.edu@gmail.com
    Co-authored-by: Jan Bárta 45016873+jbartadev@users.noreply.github.com

    下载附件