github-featured/apify--crawlee
2
0
Fork 0
Code Issues Pull Requests Deployments Models Agent Notes Packages Projects Releases Wiki Activity

Commit Graph

  • aae01cad2b Do not ignore symbols that are in fact part of the public API, remove obsolete @private decorators continued-public-surface-minimization Jan Buchar 2026-08-21 18:12:34 +02:00
  • 0c240ef23a Add a guide about the public API export and our BC guarantees Jan Buchar 2026-08-20 21:29:39 +02:00
  • 3ea0c1bb79 refactor!: Make BasicCrawler.requestManager read-only Jan Buchar 2026-08-19 22:26:47 +02:00
  • 6a73a03137 refactor!: Collapse the context pipeline seam into contextPipelineBuilder Jan Buchar 2026-08-19 11:12:23 +02:00
  • 32cd4aa31e Handle members that can be marked private/internal without additional changes Jan Buchar 2026-08-18 14:48:18 +02:00
  • 8cb6a5399c Update tests fix-apify-rq-purging Jan Buchar 2026-08-20 12:04:14 +02:00
  • 4f66a3b24b fix: Handle purge() calls with backends that don't implement the operation Jan Buchar 2026-08-20 10:27:23 +02:00
  • 7b08dbacdb refactor!: let the crawler place the throttler and decide the purge on ownership request-manager-backpressure Jan Buchar 2026-08-20 21:09:21 +02:00
  • 3989da55ce refactor!: unify per-domain pacing into IRequestManager.recordPacingSignal() Jan Buchar 2026-08-20 21:08:51 +02:00
  • 843e5f772f refactor!: collapse the readiness protocol into IRequestLoader.readiness() Jan Buchar 2026-08-20 21:03:55 +02:00
  • 69a824b77a refactor(core): retire TS-private test seams and convert to native private fields (#4043) master Harry 2026-08-21 15:57:50 +02:00
  • 73ee2b87db fix: Deflake tests under high concurrency (#4059) Jan Buchar 2026-08-21 14:28:59 +02:00
  • 3c2dd5cc37 chore: use typescript 7 for the packages, keep 6 for the docs build (#4058) Martin Adámek 2026-08-21 09:07:53 +02:00
  • d57a413371 ci: pin third-party actions to commit SHAs (#4051) Martin Adámek 2026-08-20 14:30:29 +02:00
  • f5290011a8 fix(otel): contain telemetry failures and cover the module hook claude/crawlee-pr-3361-review-9qzfn5 Claude 2026-08-20 11:38:52 +00:00
  • b1e03f911f chore: rebuild lockfile from master to drop a stray dependency downgrade Claude 2026-08-20 09:32:45 +00:00
  • 0d12dcb1f2 fix(otel): unbreak the docs build and restore the public API report Josef Prochazka 2026-08-20 09:06:28 +00:00
  • 6d77689eb6 chore(deps): security bumps + in-range freshening (#4052) apify-deps[bot] 2026-08-20 10:25:42 +02:00
  • 278f67b213 ci: cap Build & Test at 30 minutes ci/build-test-timeout Ondra Urban 2026-08-20 10:17:17 +02:00
  • 24488e2d0f fix(otel): add the public API report and document the working hook setup Josef Prochazka 2026-08-20 07:58:27 +00:00
  • 6325bc32f5 chore(deps): keep oxlint on 1.62 (drop opportunistic lint-tooling bump) chore/deps-sweep-2 apify-deps[bot] 2026-08-20 00:48:02 +00:00
  • 1b8dace6e2 chore(deps): security bumps + in-range freshening apify-deps[bot] 2026-08-20 00:34:30 +00:00
  • 8516a9ec13 chore: drop the pnpm allow-build flag chore/extract-api-extractor Jindřich Bär 2026-08-19 19:34:09 +02:00
  • b0e828ae4c chore(deps): bump pnpm to 11.22.0 (#4050) Martin Adámek 2026-08-19 19:20:12 +02:00
  • b863909574 fix: use packaged templates instead of mutable remote fetches aisle-fix-mutable-remote-template-code-execution-7e8e68fc weareaisle-apify-app 2026-08-19 17:09:24 +00:00
  • f19a1eff48 Merge master and port @crawlee/otel to Crawlee v4 Josef Prochazka 2026-08-19 14:46:19 +00:00
  • 9918489820 chore(deps): bump nanoid from 5.1.9 to 5.1.16 (#4046) dependabot[bot] 2026-08-19 14:17:00 +02:00
  • a0719843c2 fix(otel): review fixes for the OpenTelemetry instrumentation Josef Prochazka 2026-08-19 11:55:26 +00:00
  • 452afb1408 docs: add a playbook for switching master to the next major (#4049) Martin Adámek 2026-08-19 12:56:13 +02:00
  • b04de353bd chore(deps): update dependency puppeteer to v25.8.0 (#4048) Martin Adámek 2026-08-19 09:41:26 +02:00
  • 249812cb37 chore(e2e): update playwright-chrome image tags, keep camoufox fixture on 0.11/1.60 chore/v4-bump-playwright-camoufox Martin Adámek 2026-08-13 08:14:30 +02:00
  • 4a481a23aa chore: bump playwright to 1.62.1 and camoufox-js to ^0.12.0 Martin Adámek 2026-08-12 23:23:12 +02:00
  • c48b8bfa04 fix(playwright): stop pinning the first page in reused Cloudflare challenge options fix/cloudflare-challenge-hook-v4 Martin Adámek 2026-08-13 08:59:05 +02:00
  • 88eff84d38 fix(playwright): poll for late-rendered Cloudflare challenge markup Martin Adámek 2026-08-13 08:27:23 +02:00
  • 8fe51f63eb chore: move the api surface extractor to its own repo Jindřich Bär 2026-08-18 14:17:27 +02:00
  • 2137c701e5 chore(ci): point the release workflows at master for the v4 line Martin Adámek 2026-08-18 18:05:49 +02:00
  • a1512cf367 chore: restore the typed schema-router overloads on createFileRouter Martin Adámek 2026-08-18 18:05:49 +02:00
  • 2766e87443 docs: drop the 4.0 snapshot in favor of serving v4 docs as the next version Martin Adámek 2026-08-18 18:05:49 +02:00
  • 80d7010f06 chore: Add oxlint plugin to warn about TS-private class members (#4035) Jan Buchar 2026-08-17 15:34:07 +02:00
  • ceb936af2a refactor!: Continue public API minimization (#4036) Jan Buchar 2026-08-17 15:22:55 +02:00
  • 6188678305 chore(e2e): retry the platform statistics lookup instead of crashing on a read-after-write race Martin Adámek 2026-08-14 17:23:05 +02:00
  • ffd240dc9d docs: warn against --omit=optional in v4 upgrading guide (#4034) Jindřich Bär 2026-08-14 14:43:44 +02:00
  • a7b2f1c74b feat: Add Symbol.asyncDispose to disposable services for DX (#4032) Jan Buchar 2026-08-14 12:21:01 +02:00
  • 7aea1f5c37 chore(e2e): keep 403 responses retryable in the camoufox-cloudflare fixture Martin Adámek 2026-08-13 00:59:12 +02:00
  • af00b2a524 docs: pin 3.18 API links in content carried over from master Martin Adámek 2026-08-12 23:37:42 +02:00
  • bcc97ffab7 chore(e2e): pair camoufox-js ^0.11.0 with playwright 1.60.0 in the cloudflare fixture Martin Adámek 2026-08-12 23:03:44 +02:00
  • 5a7d0b9ca2 chore(e2e): repair E2E suite for the enqueueLinks split and the zod catalog dependency Martin Adámek 2026-08-12 22:41:11 +02:00
  • c324386fde chore: fix formatting and regenerate public API reports after the rebase Martin Adámek 2026-08-12 22:16:02 +02:00
  • 4bca1e65f7 chore(test): use crawler.statistics in the maxRequestsPerCrawl budget tests Martin Adámek 2026-08-12 21:46:37 +02:00
  • 41a2584d43 refactor!: Rename BasicCrawler.stats to statistics (#4028) Jan Buchar 2026-08-12 21:39:50 +02:00
  • 39148e6f1a refactor!: replace browserPoolOptions with browser pool factories (#4026) Jan Buchar 2026-08-12 20:23:26 +02:00
  • 839a9b2d09 chore: re-apply master-side fixes over the enqueueLinks split (#4010) Martin Adámek 2026-08-12 20:03:34 +02:00
  • e00e8ce84f feat: Accept custom extensions in the IStatistics interface (#3982) Jan Buchar 2026-08-12 18:38:21 +02:00
  • d50177c014 feat: Improve sameDomainDelaySecs implementation via ThrottlingRequestManager (#4017) Jan Buchar 2026-08-12 18:17:47 +02:00
  • 8e3194b91a chore: Rename client load signal to storageBackend (#4023) Jan Buchar 2026-08-12 18:08:48 +02:00
  • 6bcd06e668 feat: split enqueueLinks into extractLinks + addRequests (#4010) Jindřich Bär 2026-08-12 18:06:31 +02:00
  • 375b08c9d5 chore(test): finish adapting master-carried tests to v4 APIs Martin Adámek 2026-08-12 19:47:01 +02:00
  • 75412fce14 chore(test): adapt master-carried tests to v4 APIs after rebase Martin Adámek 2026-08-12 19:38:26 +02:00
  • bd20a49f8a chore(utils): re-add tldts dependency for enqueue-strategy URL filtering Martin Adámek 2026-08-12 19:25:39 +02:00
  • bbd72d8ed2 chore: restore @ts-ignore on robotsParser calls under nodenext resolution Martin Adámek 2026-08-12 19:24:43 +02:00
  • 3ce18eac8e chore(templates): keep camoufox-js ^0.11.0 paired with playwright 1.60.0 in the camoufox template Martin Adámek 2026-08-12 19:14:25 +02:00
  • dd6e3b9929 chore!: unify argument validation to zod (#3935) Vlad Frangu 2026-08-12 17:59:23 +03:00
  • ea9da4c1c2 test: adapt E2E suite to v4 and fix regressions it caught (#4012) Martin Adámek 2026-08-12 16:27:21 +02:00
  • f08e8d6a8c refactor: Use RecoverableState to implement Statistics persistence (#4014) Jan Buchar 2026-08-12 15:18:43 +02:00
  • 0d880420c7 feat: Extend RecoverableState API to accommodate future adopters (#4006) Jan Buchar 2026-08-12 15:05:39 +02:00
  • 4dbce0a6b0 fix: Fix purgeOnStart on aliased storages (#4013) Jan Buchar 2026-08-12 14:05:25 +02:00
  • bb2f26743f chore: support RC releases in the publish workflow (#4009) Martin Adámek 2026-08-12 13:09:11 +02:00
  • d48cd5b855 chore: wire the ignoreTlsErrors crawler option (formerly ignoreSslErrors) through to the HTTP client (#4011) Martin Adámek 2026-08-12 11:00:23 +02:00
  • 400fadf408 chore: convert const enums to regular enums (#4008) Vlad Frangu 2026-08-12 10:18:10 +03:00
  • ea5b8c22df perf(http-client): lazy-load tough-cookie (#4007) Jindřich Bär 2026-08-11 10:53:33 +02:00
  • e09444c88f perf(utils): lazily import sax (#4001) Jindřich Bär 2026-08-10 17:10:26 +02:00
  • 7bca635795 feat: implement per-domain request throttling (ThrottlingRequestManager) (#3741) Harry 2026-08-10 11:54:24 +02:00
  • 6a94b86a38 feat(utils): split exports into public and /internal entry points (#3991) Jindřich Bär 2026-08-07 14:37:12 +02:00
  • 60c4b55491 perf: lazy-load file-type and csv-stringify (#3989) Jindřich Bär 2026-08-07 13:19:48 +02:00
  • d95cf20a67 fix(browser-pool): make PlaywrightBrowser.browserType() return the browser type (#3981) Martin Adámek 2026-08-07 12:41:15 +02:00
  • dd7748d649 fix: declare own CookieJar types in @crawlee/types (#3978) Jindřich Bär 2026-08-07 12:22:20 +02:00
  • c596d23872 fix: don't ship idcac-playwright as a hard dependency (#3977) Jindřich Bär 2026-08-07 10:19:04 +02:00
  • 4d1407208b refactor!: use native private class fields and remove underscore prefixes (#3980) Martin Adámek 2026-08-06 17:37:57 +02:00
  • 80effc1952 docs: Document LoadSignal customization for ConcurrencySystem (#3976) Jan Buchar 2026-08-06 14:59:18 +02:00
  • bb6fc1d2c6 feat!: Transactional storage access (#3953) Jan Buchar 2026-08-06 14:35:05 +02:00
  • 28e24b4152 feat: add user data generics to browser navigation hooks (#3970) Martin Adámek 2026-08-06 12:24:48 +02:00
  • d915e537dc refactor!: Accept a pre-configured Statistics instance in crawlers instead of an option bag (#3966) Jan Buchar 2026-08-06 11:14:50 +02:00
  • 6a1ea155bd test: fix flaky gotoOptions assertion in browser crawler test (#3973) Martin Adámek 2026-08-06 11:02:22 +02:00
  • a99f894c17 docs: Update SessionPool section in upgrading guide (#3974) Jan Buchar 2026-08-06 10:53:11 +02:00
  • f2ef8993ea docs: document removal of gotoOptions argument from browser navigation hooks (#3972) Martin Adámek 2026-08-06 10:18:58 +02:00
  • cdab1b175c perf: lazy-load cheerio in playwright/puppeteer parseWithCheerio (#3968) Jindřich Bär 2026-08-06 09:59:11 +02:00
  • 7373844c21 refactor: drop fs-extra dependency (#3969) Jindřich Bär 2026-08-06 09:58:44 +02:00
  • 1e482e4222 docs: Reorganize the v4 upgrading guide (#3967) Jan Buchar 2026-08-05 21:01:17 +02:00
  • 7c90ca0de6 chore: Add missing upgrading guide entries (#3965) Jan Buchar 2026-08-05 15:26:51 +02:00
  • 3adf3ae4f1 feat: simplify enqueueLinks interface (#3533) Richard Solar 2026-08-05 15:11:35 +02:00
  • d53ab9b6fa chore: Improve public API snapshot generation (#3964) Jan Buchar 2026-08-05 13:53:31 +02:00
  • f2f43976ac refactor: Remove unnecessary retries when enqueueing new requests (#3962) Jan Buchar 2026-08-05 12:27:40 +02:00
  • f01b620352 feat: redesign request timeouts and allow per-route overrides (#3860) Martin Adámek 2026-08-05 10:55:28 +02:00
  • fb392227a9 fix(browser-crawler): persist session cookies after requestHandler (#3939) Felix-Ayush 2026-08-04 16:17:30 +05:30
  • 9667ab4017 fix(core): preserve subclass overrides when binding methods to a scoped service locator (#3951) Martin Adámek 2026-08-03 15:04:32 +02:00
  • 6bf2dc5280 refactor: move async iterable helpers to @crawlee/core (#3930) Jindřich Bär 2026-08-03 14:18:26 +02:00
  • 46313998b8 fix(core): log a warning on malformed cookie fragments in mergeCookies (#3850) Jindřich Bär 2026-08-03 14:11:00 +02:00
  • 83451f216e refactor!: Make AutoscaledPool internal (#3947) Jan Buchar 2026-08-03 13:09:12 +02:00
  • 3ac7fc41c6 refactor: move buffer and stream helpers to @crawlee/core (#3928) Jindřich Bär 2026-07-31 17:11:10 +02:00
  • fc2930f2ae refactor: drop re-exported types from @crawlee/utils (#3931) Jindřich Bär 2026-07-31 16:35:45 +02:00