diff --git a/sources/academy/index.mdx b/sources/academy/index.mdx index 33b6107ae8..95cd6fd13f 100644 --- a/sources/academy/index.mdx +++ b/sources/academy/index.mdx @@ -6,7 +6,7 @@ slug: / displayed_sidebar: courses hide_table_of_contents: true --- -import AcademyCard from "@site/src/components/AcademyCard"; +import Card from "@site/src/components/Card"; import CardGrid from "@site/src/components/CardGrid"; import homepageContent from "./homepage_content.json"; @@ -20,7 +20,7 @@ Learn everything about web scraping and automation with our free courses that wi { sections.map((section) => - Remember to check out our [API documentation](/api/v2) with examples in different languages and a live API console. We also recommend testing the API with a nice desktop client like [Postman](https://www.getpostman.com/) or [Insomnia](https://insomnia.rest). @@ -24,15 +22,15 @@ If the actor being run via API takes 5 minutes or less to complete a typical run ## Run an Actor or task {#run-an-actor-or-task} -> If you are unsure about the differences between an Actor and a task, you can read about them in the [tasks](../actors/running/tasks.md) documentation. In brief, tasks are just pre-configured inputs for Actors. +> If you are unsure about the differences between an Actor and a task, you can read about them in the [tasks](/platform/actors/running/tasks) documentation. In brief, tasks are just pre-configured inputs for Actors. The API endpoints and usage (for both sync and async) for [Actors](/api/v2#/reference/actors/run-collection/run-actor) and [tasks](/api/v2#/reference/actor-tasks/run-collection/run-task) are essentially the same. -To run, or **call**, an actor/task, you will need a few things: +To run, or **call**, an Actor/task, you will need a few things: -- The name or ID of the actor/task. The name looks like `username~actorName` or `username~taskName`. The ID can be retrieved on the **Settings** page of the actor/task. +- The name or ID of the Actor/task. The name looks like `username~actorName` or `username~taskName`. The ID can be retrieved on the **Settings** page of the Actor/task. -- Your [API token](../integrations/index.md), which you can find on the **Integrations** page in the [Apify Console](https://console.apify.com/account?tab=integrations) (make sure it does not get leaked anywhere!). +- Your [API token](/platform/integrations), which you can find on the **Integrations** page in [Apify Console](https://console.apify.com/account?tab=integrations) (do not share it with anyone!). - Possibly an input, which is passed in JSON format as the request's **body**. @@ -60,7 +58,7 @@ We can also add settings for the actor (which will override the default settings https://api.apify.com/v2/acts/ACTOR_NAME_OR_ID/runs?token=YOUR_TOKEN&memory=8192&build=beta ``` -This works nearly identically for both actors and tasks; however, for tasks there is no reason to specify a [`build`](../actors/development/builds.md) parameter, as a task already has only one specific actor build which cannot be changed with query parameters. +This works in almost exactly the same way for both Actors and tasks; however, for tasks, there is no reason to specify a [`build`](/platform/actors/development/builds) parameter, as a task already has only one specific Actor build which cannot be changed with query parameters. ### Input JSON {#input-json} @@ -94,7 +92,7 @@ If your synchronous run exceeds the 5-minute time limit, the response will be a ### Synchronous runs with dataset output {#synchronous-runs-with-dataset-output} -Most actor runs will store their data in the default [dataset](../storage/dataset.md). The Apify API provides **run-sync-get-dataset-items** endpoints for [actors](/api/v2#/reference/actors/run-actor-synchronously-and-get-dataset-items/run-actor-synchronously-with-input-and-get-dataset-items) and [tasks](/api/v2#/reference/actor-tasks/run-task-synchronously-and-get-dataset-items/run-task-synchronously-and-get-dataset-items-(post)), which allow you to run an actor and receive the items from the default dataset once the run has completed. +Most Actor runs will store their data in the default [dataset](/platform/storage/dataset). The Apify API provides **run-sync-get-dataset-items** endpoints for [actors](/api/v2#/reference/actors/run-actor-synchronously-and-get-dataset-items/run-actor-synchronously-with-input-and-get-dataset-items) and [tasks](/api/v2#/reference/actor-tasks/run-task-synchronously-and-get-dataset-items/run-task-synchronously-and-get-dataset-items-(post)), which allow you to run an Actor and receive the items from the default dataset once the run has finished. Here is a simple Node.js example of calling a task via the API and logging the dataset items to the console: @@ -131,7 +129,7 @@ items.forEach((item) => { ### Synchronous runs with key-value store output {#synchronous-runs-with-key-value-store-output} -[Key-value stores](../storage/key_value_store.md) are useful for storing files like images, HTML snapshots, or JSON data. The Apify API provides **run-sync** endpoints for [actors](/api/v2#/reference/actors/run-actor-synchronously/with-input) and [tasks](/api/v2#/reference/actor-tasks/run-task-synchronously/run-task-synchronously), which allow you to run a specific task and receive the output. By default, they return the `OUTPUT` record from the default key-value store. +[Key-value stores](/platform/storage/key-value-store) are useful for storing files like images, HTML snapshots, or JSON data. The Apify API provides **run-sync** endpoints for [actors](/api/v2#/reference/actors/run-actor-synchronously/with-input) and [tasks](/api/v2#/reference/actor-tasks/run-task-synchronously/run-task-synchronously), which allow you to run a specific task and receive the output. By default, they return the `OUTPUT` record from the default key-value store. > For more detailed information, check the [API reference](/api/v2#/reference/actors/run-actor-synchronously-and-get-dataset-items/run-actor-synchronously-with-input-and-get-dataset-items). @@ -165,13 +163,13 @@ Once again, the final response will be the **run info object**; however, now its #### Webhooks {#webhooks} -If you have a server, [webhooks](../integrations/webhooks/index.md) are the most elegant and flexible solution for integrations with Apify. You can simply set up a webhook for any actor or task, and that webhook will send a POST request to your server after an [event](../integrations/webhooks/events.md) has occurred. +If you have a server, [webhooks](/platform/integrations/webhooks) are the most elegant and flexible solution for integrations with Apify. You can simply set up a webhook for any Actor or task, and that webhook will send a POST request to your server after an [event](/platform/integrations/webhooks/events) has occurred. Usually, this event is a successfully finished run, but you can also set a different webhook for failed runs, etc. ![Webhook example](./images/webhook.png) -The webhook will send you a pretty complicated [JSON object](../integrations/webhooks/actions.md), but usually you are only interested in the `resource` object within the response, which is essentially just the **run info** JSON from the previous sections. We can leave the payload template as is as for our example use case, since it is what we need. +The webhook will send you a pretty complicated [JSON object](/platform/integrations/webhooks/actions), but usually, you would only be interested in the `resource` object within the response, which is essentially just the **run info** JSON from the previous sections. We can leave the payload template as is for our example since it is all we need. Once your server receives this request from the webhook, you know that the event happened, and you can ask for the complete data. @@ -195,7 +193,7 @@ Once a status of `SUCCEEDED` or `FAILED` has been received, we know the run has Unless you used the [synchronous call](#synchronous-flow) mentioned above, you will have to make one additional request to the API to retrieve the data. -The **run info** JSON also contains the IDs of the default [dataset](../storage/dataset.md) and [key-value store](../storage/key_value_store.md) that are allocated separately for each run, which is usually everything you need. The fields are called `defaultDatasetId` and `defaultKeyValueStoreId`. +The **run info** JSON also contains the IDs of the default [dataset](/platform/storage/dataset) and [key-value store](/platform/storage/key-value-store) that are allocated separately for each run, which is usually everything you need. The fields are called `defaultDatasetId` and `defaultKeyValueStoreId`. #### Retrieving a dataset {#retrieve-a-dataset} @@ -219,7 +217,7 @@ https://api.apify.com/v2/datasets/DATASET_ID/items?format=csv&offset=250000 #### Retrieving a key-value store {#retrieve-a-key-value-store} -> [Key-value stores](../storage/key_value_store.md) are mainly useful if you have a single output or any kind of files that cannot be [stringified](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify) (such as images or PDFs). +> [Key-value stores](/platform/storage/key-value-store) are mainly useful if you have a single output or any kind of files that cannot be [stringified](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify) (such as images or PDFs). When you want to retrieve something from a key-value store, the `defaultKeyValueStoreId` is _not_ enough. You also need to know the name (or **key**) of the record you want to retrieve. diff --git a/sources/academy/tutorials/apify_scrapers/index.md b/sources/academy/tutorials/apify_scrapers/index.md index 0c493b1c6c..c6cd9af8b9 100644 --- a/sources/academy/tutorials/apify_scrapers/index.md +++ b/sources/academy/tutorials/apify_scrapers/index.md @@ -1,7 +1,7 @@ --- title: Apify scrapers description: Discover Apify's ready-made web scraping and automation tools. Compare Web Scraper, Cheerio Scraper and Puppeteer Scraper to decide which is right for you. -sidebar_position: 3.2 +sidebar_position: 13.2 slug: /apify-scrapers --- @@ -42,4 +42,3 @@ Puppeteer Scraper is the most powerful scraper tool in our arsenal (aside from d Puppeteer is a Node.js library, so knowledge of Node.js and its paradigms is expected when working with Puppeteer Scraper. [Visit the Puppeteer Scraper tutorial to get started!](./puppeteer_scraper.md) - diff --git a/sources/platform/about.md b/sources/platform/about.md deleted file mode 100644 index f4189a4695..0000000000 --- a/sources/platform/about.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -title: About -description: Learn about what Apify is and how it can help you. Get familiar with the platform and take you first steps in using actors. -sidebar_position: 2 -category: guides -slug: /about ---- - -# What is Apify? {#what-is-apify} - -**Learn about what Apify is and how it can help you. Get familiar with the platform and take you first steps in using actors.** - ---- - -Apify is an online platform specializing in [web scraping](/academy/web-scraping-for-beginners), data processing and [robotic process automation](/academy/concepts/robotic-process-automation). - -[Apify Store](https://apify.com/store) includes a range of ready-made tools such as **Booking Scraper** ([dtrungtin/booking-scraper](https://apify.com/dtrungtin/booking-scraper)), **SEO Audit Tool** ([drobnikj/seo-audit-tool](https://apify.com/drobnikj/seo-audit-tool)), or the **Google Sheets Import & Export** ([lukaskrivka/google-sheets](https://apify.com/lukaskrivka/google-sheets)) tool. You can try all of these for free right now. - -If your needs are more specific, you can order a custom solution from [us](https://apify.com/enterprise). - -## Get started {#get-started} - -Check out [this video](https://www.youtube.com/watch?v=BsidLZKdYWQ) on how to use our **Amazon Scraper** ([vaclavrut/amazon-crawler](https://apify.com/vaclavrut/amazon-crawler)). It introduces how web scraping can help your business and serves as a foundation on how to start using other ready-made actors. - -For an overview of all of Apify's features, take a [tour of the platform](https://www.youtube.com/watch?v=nn-bCRvhNUM). - -### Tutorials {#tutorials} - -When you're ready to go more in depth, visit our [tutorials](./tutorials/index.md) page, which guides you through using our ready-made scrapers. - -## How does Apify work? {#how-does-apify-work} - -Almost everything we do is centered around [actors](./actors/index.md). Actors are cloud programs that can scrape web pages, process data, automate workflows or anything else you can imagine. - -You can learn more about actors from their [documentation](./actors/index.md). diff --git a/sources/platform/access_rights/index.md b/sources/platform/access_rights/index.md deleted file mode 100644 index 7012d96558..0000000000 --- a/sources/platform/access_rights/index.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -title: Access rights -description: Manage permissions for organizations or private resources such as actors, actor runs and storages. Allow other users to read, run, modify or build new versions. -sidebar_position: 12 -category: platform -slug: /access-rights ---- - -# Access rights {#access-rights} - -**Manage permissions for organizations or private resources such as actors, actor runs and storages. Allow other users to read, run, modify or build new versions.** - ---- - -By default, each system resource (actor, key-value store, run, ...) you create is only available to you, the owner. There are three ways you can grant access to your resources: the [access rights](#granting-access-rights) system, using the [organization account](#organization-account) or by [publishing your actor](#publishing-in-apify-store) in [Apify Store](https://apify.com/store). - -## Granting access rights {#granting-access-rights} - -You can easily and securely share your own resources - actors, tasks, key-value stores, datasets and request queues using a [granular](https://www.google.com/search?client=firefox-b-d&q=define+granular+permissions) permissions system. This enables you to, for example, allow your colleague to run an [actor](../actors/index.md) or view a [dataset](../storage/dataset.md), but not to modify them. - -You can also grant permission to update an actor and build a new version. -[Storages](../storage/index.md) (key-value stores, request queues and datasets) are sharable the same way with, for example, just **read** permission, or a combination of both **read** and **write** permissions. - -> To be able to grant access rights to another user you must have a **username** set in [account settings](https://console.apify.com/account?tab=settings). - -To share an actor, task, key-value store, request queue, or dataset, click on the **Actions** button in the top-right corner and select **Share**. You can add a collaborator using their **user ID**, **email**, or **username**. Once you have added the collaborator, you can configure their permissions. - -![Access rights configuration](../images/access-rights.png) - -## Organization account {#organization-account} - -Apify's organization account allows you to grant role-specific access permissions to members of your team. For more details, see the organization account [documentation](./organization_account/index.md). - -## Publishing in Apify Store {#publishing-in-apify-store} - -Another way to share your Actor is to [publish](./actors/publishing/publish) it in [Apify Store](https://apify.com/store), which means that **any user will be able to view its source code and run it**. - -> When you publish an actor, you give every Apify user the **read** and **run** permissions. - -However, **without the relevant permissions (e.g. write, build, delete), no one except the author can modify an actor**. - -If a user runs the actor, it gets billed to their account, and the actor run will be available only to them. - -To publish an actor, open it in the [Apify Console](https://console.apify.com), go to the **Publication** tab and follow the instructions below. - -![Publishing your actor](../images/publication.png) - -## Permissions {#permissions} - -Below are the permissions you can grant via the [access rights](#granting-access-rights) system, [organization account](./organization_account/index.md), and [publishing in Apify Store](./actors/publishing/publish). - -* The access rights system allows you to manage a **specific** user's permissions for a **specific** resource. -* The organization account allows you to manage **each** member's permissions for **all** of an organization's resources. -* By publishing your actor in Apify Store, you grant **all** of its users **read** (view its source code) and **run** [permissions](./list_of_permissions.md). - -For a full list of permissions available for resources on the Apify platform, see the [List of permissions](./list_of_permissions.md). diff --git a/sources/platform/actors/development/builds.md b/sources/platform/actors/development/builds.md index 7fb8f1fdcd..8fdab5a9b1 100644 --- a/sources/platform/actors/development/builds.md +++ b/sources/platform/actors/development/builds.md @@ -43,4 +43,4 @@ Each build starts with the initial status **READY** and goes through one or more ## Sharing {#sharing} -You can allow other Apify users to view your actor builds using the [access rights](../../access_rights/index.md) system. [See the full list of permissions](../../access_rights/list_of_permissions.md). +You can allow other Apify users to view your Actor builds by using the [access rights](../../collaboration/index.md) system. [See the full list of permissions](../../collaboration/list_of_permissions.md). diff --git a/sources/platform/actors/development/continuous_integration.md b/sources/platform/actors/development/continuous_integration.md index d19dcd7a3c..9b70ad72df 100644 --- a/sources/platform/actors/development/continuous_integration.md +++ b/sources/platform/actors/development/continuous_integration.md @@ -83,7 +83,7 @@ jobs: ## Prerequisites -You will need an [actor](../../tutorials/quick_start.md), a GitHub repo for it, and your Apify token. +To follow along, you will need a GitHub repository containing your [Actor](../index.md) code and your Apify API token. [Find your Apify token in the Apify Console](https://console.apify.com/account#/integrations). diff --git a/sources/platform/actors/development/images/home/actor-build.png b/sources/platform/actors/development/images/home/actor-build.png new file mode 100644 index 0000000000..ce652454b1 Binary files /dev/null and b/sources/platform/actors/development/images/home/actor-build.png differ diff --git a/sources/platform/actors/development/images/home/actor-create-button.png b/sources/platform/actors/development/images/home/actor-create-button.png new file mode 100644 index 0000000000..0df4362baa Binary files /dev/null and b/sources/platform/actors/development/images/home/actor-create-button.png differ diff --git a/sources/platform/actors/development/images/home/actor-create-templates.png b/sources/platform/actors/development/images/home/actor-create-templates.png new file mode 100644 index 0000000000..84aee549a4 Binary files /dev/null and b/sources/platform/actors/development/images/home/actor-create-templates.png differ diff --git a/sources/platform/actors/development/images/home/actor-create.gif b/sources/platform/actors/development/images/home/actor-create.gif new file mode 100644 index 0000000000..1da1e94232 Binary files /dev/null and b/sources/platform/actors/development/images/home/actor-create.gif differ diff --git a/sources/platform/actors/development/images/home/actor-input.png b/sources/platform/actors/development/images/home/actor-input.png new file mode 100644 index 0000000000..6315880f25 Binary files /dev/null and b/sources/platform/actors/development/images/home/actor-input.png differ diff --git a/sources/platform/actors/development/images/home/actor-local-code.png b/sources/platform/actors/development/images/home/actor-local-code.png new file mode 100644 index 0000000000..974ae5d8ca Binary files /dev/null and b/sources/platform/actors/development/images/home/actor-local-code.png differ diff --git a/sources/platform/actors/development/images/home/actor-local-run.png b/sources/platform/actors/development/images/home/actor-local-run.png new file mode 100644 index 0000000000..a7e162bd40 Binary files /dev/null and b/sources/platform/actors/development/images/home/actor-local-run.png differ diff --git a/sources/platform/actors/development/images/home/actor-run.png b/sources/platform/actors/development/images/home/actor-run.png new file mode 100644 index 0000000000..4c4121f022 Binary files /dev/null and b/sources/platform/actors/development/images/home/actor-run.png differ diff --git a/sources/platform/actors/development/images/home/actor-source-code.png b/sources/platform/actors/development/images/home/actor-source-code.png new file mode 100644 index 0000000000..2858c996b1 Binary files /dev/null and b/sources/platform/actors/development/images/home/actor-source-code.png differ diff --git a/sources/platform/actors/development/index.md b/sources/platform/actors/development/index.md index 77fabf1e83..bdf5e4e078 100644 --- a/sources/platform/actors/development/index.md +++ b/sources/platform/actors/development/index.md @@ -11,25 +11,142 @@ slug: /actors/development --- -In this section, you can find documentation on creating your own actors using the Apify platform. +There are two ways how to create an Actor: -**If you're interested in building paid actors, contact [community@apify.com](mailto:community@apify.com) to receive the guide.** +- Using the **Web IDE** in [Apify Console](https://console.apify.com). This is the fastest way to kick-start your Actor development and try out the Apify platform. +- Develop your Actor **locally** and only deploy to the Apify platform when it is production ready. This way, you benefit from your local setup for a better development and debugging experience. -## Before you begin +> Before you start building your own Actor, try out a couple of existing Actors from [Apify Store](https://apify.com/store). See the [running actors](./running) section for more information on running existing actors. -Here are some resources that will help you go from beginner to pro. If you have any questions, join other devs on our [Discord server](https://discord.gg/jyEM2PRvMU) or contact us at [community@apify.com](mailto:community@apify.com). +## Build Actor at Apify Console -- Get familiar with [the Apify platform](https://www.youtube.com/watch?v=nn-bCRvhNUM) and [actors](../index.md). -- [Check out our public wiki](https://gitlab.com/apify-public/wiki/-/wikis/home). It contains key information, best practices, and advice for building actors. -- [How to and how not to write an actor](https://gitlab.com/apify-public/wiki/-/wikis/writing-actors/how-to-write-and-not-write-an-actor). -- [Actor performance and optimization](https://gitlab.com/apify-public/wiki/-/wikis/writing-actors/actor-performance-and-optimization). -- [Help articles](https://help.apify.com/en/collections/1669782-publishing-to-apify-store). +### 1. Create the Actor -## Essential materials +> You will need an Apify account to complete this tutorial. If you don't have one, you can [sign-up for a free account](https://console.apify.com/sign-up) on the Apify website. + +After you sign in to [Apify Console](https://console.apify.com), navigate to the [Actors](https://console.apify.com/actors) section on your left panel. Then, click the **Create new** button at the top right corner of the page. + +![Create Actor](./images/home/actor-create-button.png) + +You will be redirected to a page containing various Actor development templates for popular languages such as JavaScript, TypeScript, and Python. You can choose the template that best suits your technology stack. For demonstration purposes, let's choose "PuppeteerCrawler in Javascript". + +![Templates](./images/home/actor-create-templates.png) + +You will be prompted to give your Actor a name and then redirected to your Actor configuration page. + +### 2. Explore the source code + +Now it's time to take a peek at the source code. You can see the boilerplate code that uses the [Apify SDK](https://docs.apify.com/sdk/js/) combined with [Crawlee](https://crawlee.dev/), Apify's popular open-source Node.js web scraping library. The code does a recursive crawl of the whole [apify.com](https://apify.com) website. + +> [Crawlee](https://crawlee.dev/) is an open-source Node.js web scraping and browser automation library.
+> It helps you build reliable crawlers. Fast. + +Let's run it! To run your Actor, you need to build it first, so let's press the "Build" button below the source code. + +![Actor source code](./images/home/actor-source-code.png) + +### 3. Build it + +After you press the "Build" button, the UI transitions to the "Build" tab, and you can see the progress of the build, including the Docker build log. + +![Actor build](./images/home/actor-build.png) + +Notice that the UI includes four tabs - (1) Code, (2) Build, (3) Input, and (4) Run. This represents the Actor creation flow where you need first to take the source code and build it. Once you have a build, you give it input which results in an Actor run. + +### 4. Run it + +Once the Actor is built, you can look at its input, which consists of one field - Start URL, the URL where the crawling starts. Below the input, you can adjust the run options consisting of build, timeout, and memory limit. + +![Actor input](./images/home/actor-input.png) + +You can start an Actor run by pressing the **Start** button at the bottom of the page. Once the run is created, you can view its log and other details, including the **Results** which will shortly start popping up in the output tab. You can always stop the run using the **Abort** button. + +![Actor run](./images/home/actor-run.png) + +### 5. Iterate + +Now you can get back to the source code and modify the Actor to match your needs. To learn about all the Apify platform features and best practices, continue to the following chapter of this section or visit the [Apify Academy](/academy). + +## Build Actor locally + +> Prerequisite - You need to have [Node.js](https://nodejs.org/en/) version 16 or higher with NPM installed on your computer. + +### 1. Install Apify CLI + +On macOS (or Linux), you can install the Apify CLI via the Homebrew package manager. + +```bash +brew install apify/tap/apify-cli +``` + +Otherwise, use [NPM](https://www.npmjs.com/) to install the Apify CLI. + +```bash +npm -g install apify-cli +``` + +For more installation and advanced usage information, see the [Apify CLI documentation](https://docs.apify.com/cli/). + +### 2. Create your Actor + +You can use the following command to create an Actor: + +```bash +apify create +``` + +You will be prompted to name your Actor (e.g, my-actor), and choose the programming language you would like to use (JavaScript, TypeScript, or Python). Next, you will be presented with a list of Actor development templates. After choosing your preferred template the CLI will: + +- Create a `my-actor` directory containing boilerplate code. +- Install all the project dependencies. + +![Creation](./images/home/actor-create.gif) + +Finally, switch to your newly created Actor directory: + +```bash +cd my-actor +``` + +### 3. Explore the source code in your editor + +Next, let's explore the Actor's source code. We will use the "PuppeteerCrawler in JavaScript" template code as an example, but all Actor templates follow a similar organizational pattern. The important parts are: + +- `src` directory and, namely, the `src/main.js` file, which is the actual code of an actor. +- `actor` directory containing actor's definition files such as `actor.json` and `Dockerfile`. These are important once you deploy your code to the Apify platform. +- `storage` directory containing the local emulation of [Apify Storage](../storage), namely [key-value store](../storage/key-value-store), [dataset](../storage/dataset), and [request queue](../storage/request-queue). + +![Actor source code](./images/home/actor-local-code.png) + +> The following chapters will teach you more about an actor's structure and files. + +### 4. Run it locally + +To run the Actor, call: + +```bash +apify run +``` + +Shortly after that, you will see the log of the Actor. The results will be stored in the local dataset under the `storage/dataset/default` directory. + +![Actor source code](./images/home/actor-local-run.png) + + +### 5. Deploy it to Apify Platform + +In order to push your Actor to the Apify platform, you need to first sign in to Apify with the CLI tool: + +```bash +apify login +``` + +And then, you can push your Actor under your Apify account with the following: + +```bash +apify push +``` + + +> If you successfully finished your first actor, you may consider [sharing it with other users and monetizing it](./publishing). -- [Actor input](https://gitlab.com/apify-public/wiki/-/wikis/public-actors/input). -- README: - - [How to make your README great](/academy/get-most-of-actors/actor-readme). -- [How to structure your actor's files](https://gitlab.com/apify-public/wiki/-/wikis/public-actors/structure). -- [Actor building checklist](https://gitlab.com/apify-public/wiki/-/wikis/public-actors/checklist). -- [Naming your actor](/academy/get-most-of-actors/naming-your-actor). diff --git a/sources/platform/actors/index.md b/sources/platform/actors/index.md index 0f44a58c34..7fa7a5d7f3 100644 --- a/sources/platform/actors/index.md +++ b/sources/platform/actors/index.md @@ -14,11 +14,11 @@ slug: /actors Actors are serverless cloud programs that can do almost anything a human can do in a web browser. They can do anything from small tasks like filling in forms or unsubscribing from online services all the way up to scraping and processing vast numbers of web pages. -You can use Actors [manually in Apify Console](https://console.apify.com/actors), or by using the [API](/api/v2) or [scheduler](../schedules.md). You can easily [integrate them with other apps](../integrations/index.md) and share your Actors with other Apify users via [Apify Store](https://apify.com/store) or the [access rights](./access-rights) system. +You can use Actors [manually in Apify Console](https://console.apify.com/actors), by using the [API](/api/v2) or [scheduler](../schedules.md). You can easily [integrate them with other apps](../integrations/index.md) and share your Actors with other Apify users via [Apify Store](https://apify.com/store) or [access rights](./collaboration/access-rights) system. -> New to Apify? Try actors with our [quick start](../tutorials/quick_start.md) tutorial and see the [Apify Academy's courses](/academy). +> New to Apify? Try actors with our [quick start](../index.mdx) tutorial and see the [Apify Academy's courses](/academy). -A single isolated actor consists of source code and various settings. You can think of an actor as a cloud app or service that runs on the Apify platform. The run of an actor is not limited to the lifetime of a single HTTP transaction. It can run for as long as necessary, even forever. +A single isolated Actor consists of source code and various settings. You can think of an Actor as a cloud app or service that runs on the Apify platform. The run of an Actor is not limited to the lifetime of a single HTTP transaction. It can run for as long as necessary, even forever. ## Public and private actors diff --git a/sources/platform/actors/running/images/actor-broken-links-checker.png b/sources/platform/actors/running/images/actor-broken-links-checker.png deleted file mode 100644 index 52b48cf997..0000000000 Binary files a/sources/platform/actors/running/images/actor-broken-links-checker.png and /dev/null differ diff --git a/sources/platform/actors/running/images/actor-google-maps-scraper-export.png b/sources/platform/actors/running/images/actor-google-maps-scraper-export.png new file mode 100644 index 0000000000..f37bc21549 Binary files /dev/null and b/sources/platform/actors/running/images/actor-google-maps-scraper-export.png differ diff --git a/sources/platform/actors/running/images/actor-google-maps-scraper-input.png b/sources/platform/actors/running/images/actor-google-maps-scraper-input.png new file mode 100644 index 0000000000..bc777aa75f Binary files /dev/null and b/sources/platform/actors/running/images/actor-google-maps-scraper-input.png differ diff --git a/sources/platform/actors/running/images/actor-google-maps-scraper-results.png b/sources/platform/actors/running/images/actor-google-maps-scraper-results.png new file mode 100644 index 0000000000..995e3b4f27 Binary files /dev/null and b/sources/platform/actors/running/images/actor-google-maps-scraper-results.png differ diff --git a/sources/platform/actors/running/images/actor-google-maps-scraper-run.gif b/sources/platform/actors/running/images/actor-google-maps-scraper-run.gif new file mode 100644 index 0000000000..4701bc9dbc Binary files /dev/null and b/sources/platform/actors/running/images/actor-google-maps-scraper-run.gif differ diff --git a/sources/platform/actors/running/images/actor-google-maps-scraper-running.png b/sources/platform/actors/running/images/actor-google-maps-scraper-running.png new file mode 100644 index 0000000000..7f1d62229b Binary files /dev/null and b/sources/platform/actors/running/images/actor-google-maps-scraper-running.png differ diff --git a/sources/platform/actors/running/images/input_and_output/actor-options.png b/sources/platform/actors/running/images/input_and_output/actor-options.png new file mode 100644 index 0000000000..7dc13963f5 Binary files /dev/null and b/sources/platform/actors/running/images/input_and_output/actor-options.png differ diff --git a/sources/platform/actors/running/images/store-google-maps-scraper.png b/sources/platform/actors/running/images/store-google-maps-scraper.png new file mode 100644 index 0000000000..b6fcf37376 Binary files /dev/null and b/sources/platform/actors/running/images/store-google-maps-scraper.png differ diff --git a/sources/platform/actors/running/index.md b/sources/platform/actors/running/index.md index 38bf8e8bc4..c5310f31c9 100644 --- a/sources/platform/actors/running/index.md +++ b/sources/platform/actors/running/index.md @@ -11,71 +11,89 @@ import TabItem from '@theme/TabItem'; # Running -**Start an actor from the Apify Console or via API. Learn about actor lifecycles, how to specify settings and version, provide input and resurrect finished runs.** +**In this section, you learn how to run Apify Actors using Apify Console or programmatically. You will learn about their configuration, versioning, data retention, usage, and pricing.** --- -If you have not [built](./development) your own Actor yet, then choose one from [Apify Store](https://apify.com/store). Once you have chosen an Actor, you can start it in a number of ways. +## Run your first Apify Actor -## Running from Apify Console UI +Before you can run an Actor, you have to either choose one of the existing ones from [Apify Store](https://apify.com/store) or [build your own](./development). To get started, we recommend trying out an Actor from the [Store](https://apify.com/store). Once you have chosen an Actor, you can start it in a number of ways. -One option is to start an Actor from its page in [Apify Console](https://console.apify.com/actors): +> **You will need an Apify account to complete this tutorial. If you don't have one, [complete the sign-up process](https://console.apify.com/sign-up) first. Don't worry about the price - it's free.** -![Apify developer console](./images/actor-broken-links-checker.png) +### 1. Choose your actor -After setting up an input, you can specify run options such as [build](../development/builds.md), timeout, and [memory](./usage_and_resources.md) for your Actor run. +After you sign-in to Apify Console, navigate to [Apify Store](https://console.apify.com/store). We'll pick the [Google Maps Scraper](https://console.apify.com/actors/nwua9Gu5YrADL7ZDj#/information/latest/readme): - - - - - - - - - - - - - - -
BuildTag or number of the build to run (e.g. latest or 1.2.34).
TimeoutTimeout for the actor run in seconds. Zero value means there is no timeout.
MemoryAmount of memory allocated for the actor run, in megabytes.
+![Apify Store](./images/store-google-maps-scraper.png) + +### 2. Configure it + +On the Actor's page, head over to the **Input** tab. Don't be put off by all the boxes - the Actor is pre-configured to run without any extra input. Just click the **Start** button in the bottom-left corner. + +Alternatively, you can play around with the settings to make the results more interesting for you. + +![Actor input](./images/actor-google-maps-scraper-input.png) + + +### 3. Wait for the results + +The Actor might take a while to gather its first results and finish its run. Meanwhile, let's take some time to explore the platform options: + +- There are more tabs providing you with information about the Actor run. For example, you can access the run **Log** and **Storage**. +- At the top right, you can click on the API button to explore the related API endpoints + +![Run](./images/actor-google-maps-scraper-running.png) + +### 4. Get the results + +Shortly you will see the first results popping up: + +![Actor results](./images/actor-google-maps-scraper-results.png) + + +And you can use the export button at the bottom left to export the data in multiple formats: + +![Export results](./images/actor-google-maps-scraper-export.png) + +And that's it! Now you can get back to the Actor's input, play with it, and try out more of the [Apify Actors](https://apify.com/store) or [build your own](./development). ## Running via Apify API Actors can also be invoked using the Apify API by sending an HTTP POST request to the [Run Actor](/api/v2/#/reference/actors/run-collection/run-actor) endpoint, such as: ```text -https://api.apify.com/v2/acts/apify~hello-world/runs?token= +https://api.apify.com/v2/acts/compass~crawler-google-places/runs?token= ``` An actor's input and its content type can be passed as a payload of the POST request, and additional options can be specified using URL query parameters. For more details, see the [Run Actor](/api/v2/#/reference/actors/run-collection/run-actor) section in the API reference. -> To learn more about this, read the [Run an actor or task and retrieve data via API](../tutorials/run-actor-and-retrieve-data-via-api) tutorial. +> To learn more about this, read the [Run an Actor or task and retrieve data via API](/academy/api/run-actor-and-retrieve-data-via-api) tutorial. ## Running programmatically -Actors can also be invoked programmatically from other Actors and your Python or JavaScript code: +Actors can also be invoked programmatically from your own applications or from other actors. -- JavaScript: using the [`call()`](/sdk/js/reference/class/Actor#call) function of [`Actor`](/sdk/js/reference/class/Actor) class provided by the [`apify`](/sdk/js/) NPM package. -- Python: using the [`call()`](/api/client/python/reference/class/ActorClient#call) function provided by the [`apify-client`](/api/client/python) Python package. +To start an Actor from your own application, we recommend using our API client libraries for [JavaScript](/api/client/js/reference/class/ActorClient#call) or [Python](/api/client/python/reference/class/ActorClient#call). - + ```javascript -import { Actor } from 'apify'; +import { ApifyClient } from 'apify-client'; -await Actor.init(); -// ... -const run = await Actor.call('apify/hello-world', { - message: 'Hello!', +const client = new ApifyClient({ + token: 'MY-API-TOKEN', }); -console.dir(run.output); -// ... -await Actor.exit(); +// Start the Google Maps Scraper Actor and wait for it to finish. +const actorRun = await client.actor('compass/crawler-google-places').call({ + queries: 'apify', +}); +// Fetch scraped results from the Actor's dataset. +const { items } = await client.dataset(actorRun.defaultDatasetId).listItems(); +console.dir(items); ``` @@ -84,18 +102,25 @@ await Actor.exit(); ```python +from apify_client import ApifyClient -run = apify_client.actor('apify/hello-world').call(run_input={ 'message': 'Hello!' }) -print(run['id']) +apify_client = ApifyClient('MY-API-TOKEN') +# Start the Google Maps Scraper Actor and wait for it to finish. +actor_run = apify_client.actor('compass/crawler-google-places').call( + run_input={ 'queries': 'apify' } +) +# Fetch scraped results from the Actor's dataset. +dataset_items = apify_client.dataset(actor_run['defaultDatasetId']).list_items().items +print(dataset_items) ``` -The newly started Actor runs under the same user account as the initial Actor, and therefore all resources consumed are charged to the same user account. This allows you to build more complex Actors from simpler Actors built and owned by other users. +The newly started Actor runs under the account associated with the provided `token`, and therefore all resources consumed are charged to this user account. -Internally, the `call()` function takes the user's API token from the `APIFY_TOKEN` environment variable, then it invokes the [Run actor](/api/v2/#/reference/actors/run-collection/run-actor) API endpoint, waits for the actor to finish and reads its output using the [Get record](/api/v2/#/reference/key-value-stores/record/get-record) API endpoint. +Internally, the `call()` function invokes the [Run Actor](/api/v2/#/reference/actors/run-collection/run-actor) API endpoint, waits for the Actor to finish, and reads its output using the [Get items](/api/v2/#/reference/datasets/item-collection/get-items) API endpoint. diff --git a/sources/platform/actors/running/input_and_output.md b/sources/platform/actors/running/input_and_output.md index 55b9d0060f..53aeb1f01e 100644 --- a/sources/platform/actors/running/input_and_output.md +++ b/sources/platform/actors/running/input_and_output.md @@ -27,6 +27,28 @@ When running an Actor using the [API](https://docs.apify.com/api/v2) you can pas } ``` +### Options - Build, Timeout, and Memory + +As part of the input, you can also specify run options such as [build](../development/builds.md), timeout, and [memory](./usage_and_resources.md) for your Actor run. + +![Run options](./images/input_and_output/actor-options.png) + + + + + + + + + + + + + + + +
BuildTag or number of the build to run (e.g. latest or 1.2.34).
TimeoutTimeout for the Actor run in seconds. Zero value means there is no timeout.
MemoryAmount of memory allocated for the Actor run, in megabytes.
+ ## Output While the input object provides a way to instruct Actors, an Actor can also generate an output, usually stored in its default [dataset](../../storage/dataset), but some additional files might be stored in its [key-value store](../../storage/key-value-store). Always read the Actor's README to learn more about its output. @@ -41,5 +63,5 @@ And to access all the data associated with the run, see the **storage** tab, whe ![Actor output](./images/input_and_output/actor-storage.png) -> You can also use [API](https://docs.apify.com/api/v2) to retrieve the output. To learn more about this, read the [Run an actor or task and retrieve data via API](../../tutorials/run-actor-and-retrieve-data-via-api) tutorial. +> You can also use [API](https://docs.apify.com/api/v2) to retrieve the output. To learn more about this, read the [Run an actor or task and retrieve data via API](/academy/api/run-actor-and-retrieve-data-via-api) tutorial. diff --git a/sources/platform/actors/running/runs_and_builds.md b/sources/platform/actors/running/runs_and_builds.md index 1a4d6ad468..8c26be2694 100644 --- a/sources/platform/actors/running/runs_and_builds.md +++ b/sources/platform/actors/running/runs_and_builds.md @@ -46,7 +46,7 @@ flowchart LR subgraph Transitional states RUNNING TIMING-OUT - ABORTNING + ABORTING end subgraph Terminal states @@ -60,7 +60,7 @@ flowchart LR RUNNING --> SUCCEEDED RUNNING --> FAILED RUNNING --> TIMING-OUT --> TIMED-OUT - RUNNING --> ABORTNING --> ABORTED + RUNNING --> ABORTING --> ABORTED ``` --- @@ -81,8 +81,8 @@ flowchart LR You can abort runs with the statuses **READY**, **RUNNING**, or **TIMING-OUT** in two ways: -- **Immediately** - this is the default option. -- **Gracefully** - the Actor run receives a signal about aborting via the `aborting` and `persistState` events and is force-aborted after 30 seconds. This is helpful in cases where you plan to resurrect the run later because it saves the actor's state. When resurrected, the Actor can restart where it left off. +- **Immediately** - this is the default option. The Actor process is killed immediately with no grace period. +- **Gracefully** - the Actor run receives a signal about aborting via the `aborting` event and is granted a 30-second window to finish in-progress tasks before getting aborted. This is helpful in cases where you plan to resurrect the run later because it gives the Actor a chance to persist its state. When resurrected, the Actor can restart where it left off. You can abort a run in Apify Console using the **Abort** button or via API using the [Abort run](/api/v2#/reference/actor-runs/abort-run/abort-run) endpoint. @@ -98,19 +98,19 @@ The whole process of resurrection looks as follows: Resurrection can be performed in Apify Console using the **resurrect** button or via API using the [Resurrect run](/api/v2#/reference/actors/resurrect-run) API endpoint. -> You can also adjust timeout, memory, or change actor build prior to the resurrection. This is especially helpful in a case of an error in a code of an actor as it enables you to: +> You can also adjust timeout and memory or change Actor build prior to the resurrection. This is especially helpful in a case of an error in the Actor's source code as it enables you to: > -> - Abort a broken run -> - Update the Actor's code and build the new version -> - Resurrect the run using the new build +> 1. Abort a broken run +> 2. Update the Actor's code and build the new version +> 3. Resurrect the run using the new build ### Data retention -An **Actor run** is deleted along with its default storages (key-value store, dataset, request queue) after a data retention period which is based on your [subscription plan](https://apify.com/pricing). +All **Actor runs** are deleted along with their default storages (key-value store, dataset, request queue) after the data retention period, which is based on your [subscription plan](https://apify.com/pricing). -An **Actor build** is deleted only when it is not tagged and has not been used for over 90 days. +**Actor builds** are deleted only when they are **not tagged** and have not been used for over 90 days. ## Sharing -You can share your Actor runs with other Apify users via the [access rights](../../access_rights/index.md) system. +You can share your Actor runs with other Apify users via the [access rights](../../collaboration/index.md) system. diff --git a/sources/platform/actors/running/store.md b/sources/platform/actors/running/store.md index 8ede72bb65..3dfdd0d3ee 100644 --- a/sources/platform/actors/running/store.md +++ b/sources/platform/actors/running/store.md @@ -9,10 +9,10 @@ slug: /actors/running/actors-in-store --- -> Anyone is welcome to [publish Actors](/platform/actors/publishing) in the store, and you can even [monetize your actors](https://get.apify.com/monetize-your-code) by renting them out to users of the platform. For more information about how to monetize your actor, best practices and SEO and promotion tips and tricks, head over to the [Getting the most of your public actors on Apify Store](/academy/get-most-of-actors) section of the Apify Developers Academy. +> Anyone is welcome to [publish Actors](/platform/actors/publishing) in the store, and you can even [monetize your Actors](/platform/actors/publishing/monetize) by renting them out to users of the platform. For more information about how to monetize your Actor, best practices, SEO, and promotion tips and tricks, head over to the [Getting the most of your public Actors in the Apify Store](/academy/get-most-of-actors) section of the Apify Developers Academy. -Some actors are free to use (you will be charged only for the Apify platform usage), while the others are paid with different pricing models. This page will explain to you all the approaches. +Some Actors are free to use (you will be charged only for the Apify platform usage), while others are paid with different pricing models. This page explains all the options. ## Pricing models @@ -24,11 +24,11 @@ All actors in [Apify Store](https://apify.com/store) fall into one of these thre ### Free actors -When you use a free actor you are only charged the platform usage the runs of this actor generate. [Platform usage](./usage_and_resources.md) includes components such as compute units, operations on [storages](https://docs.apify.com/platform/storage), and usage of [residential proxies](https://docs.apify.com/platform/proxy/residential-proxy) or [SERPs](). You can find the details of your actor usage on the run detail page. +When you use a free Actor, you are only charged for the platform usage that the runs of this Actor generates. [Platform usage](./usage_and_resources.md) includes components such as compute units, operations on [storages](/platform/storage), and usage of [residential proxies](/platform/proxy/residential-proxy) or [SERPs](/platform/proxy/google-serp-proxy). You can find the details of your Actor usage on the run detail page. ![Example run usage](./images/store/example_run_usage.png) -> With this model, it is quite difficult to know beforehand exactly how much usage the actor will generate when you run it, so you cannot easily estimate your costs upfront. The easiest approach to do so is to try out the actor on a limited scope, for example on a small number of pages, and evaluate the consumption. +> With this model, it's very easy to see how many platform resources each Actor run consumed, but it is quite difficult to estimate their usage beforehand. The best way to find the costs of free Actors upfront is to try out the Actor on a limited scope (for example, on a small number of pages) and evaluate the consumption. You can easily do that using our [free plan](https://apify.com/pricing). > > **For more information on platform usage cost see the [usage and resources](./usage_and_resources.md) page.** @@ -47,9 +47,7 @@ Each paid Actor has a **free trial,** where the trial's length is always shown f ![Paid Actors free trial](./images/store/paid-actors-trial.png) -After a trial, a flat monthly **Actor rental** fee that is automatically charged from your prepaid platform usage in advance for the following month. This fee goes directly to the developer and it is paid on the top of the platform usage generated by the Actor. - - +After a trial, a flat monthly **Actor rental** fee is automatically charged from your prepaid platform usage in advance for the following month. This fee goes directly to the developer, and it is paid on top of the platform usage generated by the Actor. You can read more about our motivation for releasing paid Actors in [this blog post](https://blog.apify.com/make-regular-passive-income-developing-web-automation-actors-b0392278d085/) from Apify's CEO Jan Čurn. **Can I run paid Actors via API or the Apify client?** @@ -91,7 +89,7 @@ Hence, you can find the breakdown of how much you have been charged for paid Act ### Actors paid per result -When you run an Actor that is **paid per result**, you only pay for the results that an Actor returns when you run it, and you are not charged for the underlying platform usage. Technically, **results** you are charged for are the clean items in the default [dataset](https://docs.apify.com/platform/storage/dataset) created by the Actor run. +When you run an Actor that is **paid per result**, you only pay for the successful results that an Actor returns when you run it, and you are not charged for the underlying platform usage. Technically, we validate the **results** against a schema, and you are only charged for the valid items in the [dataset](https://docs.apify.com/platform/storage/dataset) of the Actor run. > This makes it transparent and easy to estimate upfront costs. If you have any feedback or would like to ask something, please join our [Discord](https://discord.gg/qkMS6pU4cF) community and let us know! diff --git a/sources/platform/actors/running/tasks.md b/sources/platform/actors/running/tasks.md index b5a4dac26d..9bd734186b 100644 --- a/sources/platform/actors/running/tasks.md +++ b/sources/platform/actors/running/tasks.md @@ -1,13 +1,13 @@ --- title: Tasks -description: Set up configurations of your Apify Actors for simplified usage. Create multiple configurations of the same Actor tailored to various use cases. +description: Save configurations of your Apify Actors for later. Create multiple configurations of the same Actor tailored to various use cases. sidebar_position: 7.2 slug: /actors/running/tasks --- # Actor tasks -**Set up configurations of your Apify actors for simplified usage. Create multiple configurations of the same actor tailored to various use cases.** +**Save configurations of your Apify Actors for later. Create multiple configurations of the same Actor tailored to various use cases.** --- @@ -17,7 +17,7 @@ All your tasks are listed at [https://console.apify.com/actors/tasks](https://co You can create tasks both for your personal actors and for those made by someone else. Just [search Apify Store](https://console.apify.com/store) to find an actor you want to use and click the **Create new task** button. -Like any other resource, you can share your actor tasks with other Apify users via the [access rights](../../access_rights/index.md) system. +Like any other resource, you can share your Actor tasks with other Apify users via the [access rights](../../collaboration/index.md) system. ## Create @@ -41,7 +41,7 @@ To make a task easier to identify, you can give it a name, title, and descriptio ### Settings -Under the **Settings** tab of their detail page, you can grant [access rights](../../access_rights/index.md) to other Apify users. +You can grant [access rights](../../collaboration/index.md) to other Apify users under the **Settings** tab of their detail page. ## Run diff --git a/sources/platform/actors/running/usage_and_resources.md b/sources/platform/actors/running/usage_and_resources.md index 40adabe493..170763941c 100644 --- a/sources/platform/actors/running/usage_and_resources.md +++ b/sources/platform/actors/running/usage_and_resources.md @@ -44,7 +44,7 @@ Sometimes, you see the Actor’s CPU use go over 100%. This is not unusual. To h The Actor has hard disk space limited by twice the amount of memory. For example, an Actor with 1024 MB of memory will have 2048 MB of disk available. -## Requiremenets +## Requirements Actors built on top of the [Apify SDK](/sdk/js) and [Crawlee](https://crawlee.dev/) use autoscaling. This means that they will always run as efficiently as they can with the memory they have allocated. So, if you allocate 2 times more memory, the run should be 2 times faster and consume the same amount of compute units (1 * 1 = 0.5 * 2). @@ -58,7 +58,7 @@ Autoscaling only applies to solutions that run multiple tasks (URLs) for at leas [//]: # (TODO: It's pretty outdated, we now have platform credits in pricing) [//]: # (If you read that you can scrape 1000 pages of data for 1 CU and you want to scrape approximately 2 million of them monthly, that means you need 2000 CUs monthly and should [subscribe to the Business plan](https://console.apify.com/billing-new#/subscription).) -If the Actor doesn't have this information, or you want to use your own solution, just run your solution like you want to use it long term. Let's say that you want to scrape the data **every hour for the whole month**. You set up a reasonable memory allocation like 4096 MB and the whole run takes 15 minutes. That should consume 1 CU (4 \* 0.25 = 1). Now, you just need to multiply that by the number of hours in the day and by the number of days in the month, and you get an estimated usage of 720 (1 * 24 * 30) CUs monthly. +If the Actor doesn't have this information, or you want to use your own solution, just run your solution like you want to use it long term. Let's say that you want to scrape the data **every hour for the whole month**. You set up a reasonable memory allocation like 4096 MB, and the whole run takes 15 minutes. That should consume 1 CU (4 \* 0.25 = 1). Now, you just need to multiply that by the number of hours in the day and by the number of days in the month, and you get an estimated usage of 720 (1 \* 24 \* 30) CUs monthly. > [See our article on **estimating consumption**](https://help.apify.com/en/articles/3470975-how-to-estimate-compute-unit-usage-for-your-project) for more details. diff --git a/sources/platform/collaboration/access_rights.md b/sources/platform/collaboration/access_rights.md new file mode 100644 index 0000000000..4953b664f3 --- /dev/null +++ b/sources/platform/collaboration/access_rights.md @@ -0,0 +1,23 @@ +--- +title: Access rights +description: Manage permissions for your private resources such as Actors, Actor runs, and storages. Allow other users to read, run, modify, or build new versions. +sidebar_position: 12 +category: platform +slug: /collaboration/access-rights +--- + +**Manage permissions for your private resources such as Actors, Actor runs, and storages. Allow other users to read, run, modify, or build new versions.** + +--- + +You can easily and securely share your own resources - Actors, tasks, key-value stores, datasets, and request queues - with other users by using a [granular](https://www.google.com/search?client=firefox-b-d&q=define+granular+permissions) permissions system. This enables you, for example, to let your colleague run an [Actor](../actors/index.md) or view a [dataset](../storage/dataset.md) but not modify it. + +> To be able to grant access rights to another user, you must have a **username** set in [account settings](https://console.apify.com/account?tab=settings). + +You can also grant permission to update an Actor and build a new version. +[Storages](../storage/index.md) (key-value stores, request queues, and datasets) are sharable in the same way as a **read** permission or a combination of both **read** and **write** permissions. + +To share an Actor, task, key-value store, request queue, or dataset, click on the **Actions** button in the top-right corner and select **Share**. You can add a collaborator by using their **user ID**, **email**, or **username**. Once you have added the collaborator, you can configure their permissions. + +![Access rights configuration](./images/access-rights/access-rights.png) + diff --git a/sources/platform/images/access-rights.png b/sources/platform/collaboration/images/access-rights/access-rights.png similarity index 100% rename from sources/platform/images/access-rights.png rename to sources/platform/collaboration/images/access-rights/access-rights.png diff --git a/sources/platform/access_rights/images/configure-permissions.png b/sources/platform/collaboration/images/organizations/configure-permissions.png similarity index 100% rename from sources/platform/access_rights/images/configure-permissions.png rename to sources/platform/collaboration/images/organizations/configure-permissions.png diff --git a/sources/platform/access_rights/images/convert-to-organization.png b/sources/platform/collaboration/images/organizations/convert-to-organization.png similarity index 100% rename from sources/platform/access_rights/images/convert-to-organization.png rename to sources/platform/collaboration/images/organizations/convert-to-organization.png diff --git a/sources/platform/access_rights/images/create-new-org.png b/sources/platform/collaboration/images/organizations/create-new-org.png similarity index 100% rename from sources/platform/access_rights/images/create-new-org.png rename to sources/platform/collaboration/images/organizations/create-new-org.png diff --git a/sources/platform/access_rights/images/integrations.png b/sources/platform/collaboration/images/organizations/integrations.png similarity index 100% rename from sources/platform/access_rights/images/integrations.png rename to sources/platform/collaboration/images/organizations/integrations.png diff --git a/sources/platform/access_rights/images/members.png b/sources/platform/collaboration/images/organizations/members.png similarity index 100% rename from sources/platform/access_rights/images/members.png rename to sources/platform/collaboration/images/organizations/members.png diff --git a/sources/platform/access_rights/images/my-organizations.png b/sources/platform/collaboration/images/organizations/my-organizations.png similarity index 100% rename from sources/platform/access_rights/images/my-organizations.png rename to sources/platform/collaboration/images/organizations/my-organizations.png diff --git a/sources/platform/access_rights/images/roles.png b/sources/platform/collaboration/images/organizations/roles.png similarity index 100% rename from sources/platform/access_rights/images/roles.png rename to sources/platform/collaboration/images/organizations/roles.png diff --git a/sources/platform/access_rights/images/switch-to-organization.png b/sources/platform/collaboration/images/organizations/switch-to-organization.png similarity index 100% rename from sources/platform/access_rights/images/switch-to-organization.png rename to sources/platform/collaboration/images/organizations/switch-to-organization.png diff --git a/sources/platform/access_rights/images/upgrade.png b/sources/platform/collaboration/images/organizations/upgrade.png similarity index 100% rename from sources/platform/access_rights/images/upgrade.png rename to sources/platform/collaboration/images/organizations/upgrade.png diff --git a/sources/platform/collaboration/index.md b/sources/platform/collaboration/index.md new file mode 100644 index 0000000000..9a8854bfae --- /dev/null +++ b/sources/platform/collaboration/index.md @@ -0,0 +1,30 @@ +--- +title: Collaboration +description: Learn how to collaborate with other users and manage permissions for organizations or private resources such as Actors, Actor runs, and storages. +sidebar_position: 12 +category: platform +slug: /collaboration +--- + +**Learn how to collaborate with other users and manage permissions for organizations or private resources such as Actors, Actor runs, and storages.** + +--- + +By default, each system resource (Actor, key-value store, run, ...) you create is only available to you, the owner. There are three ways you can grant access to your resources: + + + + + + + + + + + + + + +
Access rightsEnables you to grant access to another user for a certain resource you own. This way, you can share results with your client, or two engineers can collaborate on developing one Actor.
Organization accountApify's organization account allows multiple engineers to collaborate on team projects with role-specific access permissions.
Publishing in Apify StoreAnother way to share your Actor with other users is to publish it in Apify Store. When publishing your Actor, you can make it a Paid Actor and get paid by the users benefiting from your tool. For more information, read the publishing and monetization section.
+ + \ No newline at end of file diff --git a/sources/platform/access_rights/list_of_permissions.md b/sources/platform/collaboration/list_of_permissions.md similarity index 95% rename from sources/platform/access_rights/list_of_permissions.md rename to sources/platform/collaboration/list_of_permissions.md index 037d9f18fe..fbb872cc0e 100644 --- a/sources/platform/access_rights/list_of_permissions.md +++ b/sources/platform/collaboration/list_of_permissions.md @@ -2,7 +2,7 @@ title: List of permissions description: Learn about the access rights you can grant to other users. See a list of all access options for Apify resources such as actors, actor runs/tasks and storage. sidebar_position: 12.2 -slug: /access-rights/list-of-permissions +slug: /collaboration/list-of-permissions --- # List of permissions @@ -108,9 +108,3 @@ Permissions that can be granted to members of organizations. To learn about the | Reset password | Reset the account's password. | | View invoices | See the account's invoices. | | Manage organization | Change the organization's settings. | - -## Custom Solutions {#custom-solutions} - -| Permission | Description | -|------------------|-----------------------------------| -| Custom Solutions | Allow access to Custom Solutions. | diff --git a/sources/platform/access_rights/organization_account/how_to_use.md b/sources/platform/collaboration/organization_account/how_to_use.md similarity index 92% rename from sources/platform/access_rights/organization_account/how_to_use.md rename to sources/platform/collaboration/organization_account/how_to_use.md index 5ac6c48ffc..d02ca67758 100644 --- a/sources/platform/access_rights/organization_account/how_to_use.md +++ b/sources/platform/collaboration/organization_account/how_to_use.md @@ -2,7 +2,7 @@ title: How to use description: Learn to use and manage your organization account using the Apify Console or API. View the organizations you are in and manage your memberships. sidebar_position: 2 -slug: /access-rights/organization-account/how-to-use +slug: /collaboration/organization-account/how-to-use --- # Using the organization account @@ -21,7 +21,7 @@ While you can't manage an organization account via [API](/api/v2), you can still You can switch into **Organization account** view using the account button in the top-left corner. -![Switch to organization account](../images/switch-to-organization.png) +![Switch to organization account](../images/organizations/switch-to-organization.png) In the menu, the account you are currently using is displayed at the top, with all the accounts you can switch to displayed below. When you need to get back to your personal account, you can just switch right back to it – no need to log in and out. @@ -35,7 +35,7 @@ You can view and manage the organizations you are a member of from the **Organiz If you want to leave an organization you own, you must first transfer ownership to someone else. -![My organizations](../images/my-organizations.png) +![My organizations](../images/organizations/my-organizations.png) ### Transfer ownership {#transfer-ownership} @@ -47,7 +47,7 @@ While you cannot manage an organization account's settings and members via API, As a member of an organization, you are assigned an [API token](../../integrations/index.md) (under the **Integrations** tab) and proxy password (click the **Proxy** button in the left menu) for accessing the Apify platform via REST API. -![Integration tokens](../images/integrations.png) +![Integration tokens](../images/organizations/integrations.png) The API tokens' functionality reflects your account's permissions in the organization, so if you only have the **read** and **run** permissions for actors, you will only be able to view and run actors via API. Only you can view your API token and password. diff --git a/sources/platform/access_rights/organization_account/index.md b/sources/platform/collaboration/organization_account/index.md similarity index 84% rename from sources/platform/access_rights/organization_account/index.md rename to sources/platform/collaboration/organization_account/index.md index 0ae4c575ed..8506120a57 100644 --- a/sources/platform/access_rights/organization_account/index.md +++ b/sources/platform/collaboration/organization_account/index.md @@ -2,7 +2,7 @@ title: Organization account description: Create a specialized account for your organization to encourage collaboration and manage permissions. Convert an existing account, or create one from scratch. sidebar_position: 12.1 -slug: /access-rights/organization-account +slug: /collaboration/organization-account --- # Organization account @@ -30,7 +30,7 @@ The organization account is available on all our plans. [Visit our pricing page] You can create a new organization by clicking the **Create new organization** button under the **Organizations** tab in your [account](https://console.apify.com/account#/myorganizations). If you want the organization to have a separate email address (used for notifications), enter it here. Otherwise, leave the **email** field empty and the owner's email will be used for notifications. -![Create a new organization](../images/create-new-org.png) +![Create a new organization](../images/organizations/create-new-org.png) **You can own up to 5 and be a member of as many organizations as you need.** @@ -48,7 +48,7 @@ An organization can't be a member of other organizations. If you want to convert Then, under the **Organizations** [tab](https://console.apify.com/account#/myorganizations), click the **Convert this user account to an organization** button. -![Convert your account to an organization](../images/convert-to-organization.png) +![Convert your account to an organization](../images/organizations/convert-to-organization.png) Next, set the organization's name and click **Convert**. @@ -60,9 +60,9 @@ For information on [adding members and assigning roles](./setup.md), see the Set Actor and task runs are billed to the account they are started from. **Always make sure you start your runs from the correct account** to avoid having an organization's runs billed to your personal account. -The Free and Personal plans are [restricted to 9 seats](https://apify.com/pricing), while the Team plan comes with 9, which you can increase later. If you are on the Team plan and want to add some more seats to the 9 it comes with, head over to your [Billing page and click on Subscriptions](https://console.apify.com/billing-new#/subscription). Click on **Upgrade**, and, in the **Add-ons** section, and add as many additional seats as you need. Then, just complete the upgrade process, and you're done. +The Free and Personal plans are [restricted to 9 seats](https://apify.com/pricing), while the Team plan comes with a minimum of 9, which you can increase later. If you are on the Team plan and want to add some more seats to the 9 it comes with, head over to your [Billing page and click on Subscriptions](https://console.apify.com/billing-new#/subscription). Click on **Upgrade**, and, in the **Add-ons** section, and add as many additional seats as you need. Then, just complete the upgrade process, and you're done. -![Adding more teams account seats](../images/upgrade.png) +![Adding more teams account seats](../images/organizations/upgrade.png) To find out about organization pricing, get in touch at [support@apify.com](mailto:support@apify.com?subject=Organization%20account%20pricing) or [visit the pricing page](https://apify.com/pricing). diff --git a/sources/platform/access_rights/organization_account/setup.md b/sources/platform/collaboration/organization_account/setup.md similarity index 90% rename from sources/platform/access_rights/organization_account/setup.md rename to sources/platform/collaboration/organization_account/setup.md index 992c309ddd..45eeee3685 100644 --- a/sources/platform/access_rights/organization_account/setup.md +++ b/sources/platform/collaboration/organization_account/setup.md @@ -2,7 +2,7 @@ title: Setup description: Configure your organization account by inviting new members and assigning their roles. Manage team members' access permissions to the organization's resources. sidebar_position: 1 -slug: /access-rights/organization-account/setup +slug: /collaboration/organization-account/setup --- # Setup @@ -21,7 +21,7 @@ You can add members to your organization in the **Members** tab. You can use the The **Members** tab allows you to view and manage your organization's members and transfer the organization's membership. -![Organization members](../images/members.png) +![Organization members](../images/organizations/members.png) ## Define roles and permissions {#define-roles-and-permissions} @@ -29,7 +29,7 @@ Roles allow you to define permissions to your organization's resources by group. To edit the permissions for each role, click on the **Configure permissions** button in the top-right corner. -![Organization roles](../images/roles.png) +![Organization roles](../images/organizations/roles.png) > Each member can only have one role to avoid conflicting permissions. @@ -37,7 +37,7 @@ You can configure individual permissions for each resource type such as actors, **Some permissions have dependencies**. For example, if someone has the **actor run** permission, it is likely they will also need the **storage write** permission, so they can store the results from their actor runs. -![Configure permissions](../images/configure-permissions.png) +![Configure permissions](../images/organizations/configure-permissions.png) [See the full list of permissions](../list_of_permissions.md) that can be granted to Apify resources. diff --git a/sources/platform/homepage_content.json b/sources/platform/homepage_content.json index c2889d1a34..2c5fc61f78 100644 --- a/sources/platform/homepage_content.json +++ b/sources/platform/homepage_content.json @@ -1,200 +1,32 @@ -{ - "Get started": [ - { - "cardItem": "What is Apify?", - "href": "/platform/about" - }, - { - "cardItem": "Quick start", - "href": "/platform/tutorials/quick-start" - }, - { - "cardItem": "Start using Apify scrapers", - "href": "/academy/apify-scrapers/getting-started" - }, - { - "cardItem": "Learn to use Web Scraper", - "href": "/academy/apify-scrapers/web-scraper" - }, - { - "cardItem": "Take a video tour of the platform", - "href": "https://www.youtube.com/watch?v=nn-bCRvhNUM", - "isExternalLink": true - }, - { - "cardItem": "Apify vocabulary", - "href": "https://help.apify.com/en/articles/3408158-apify-vocabulary", - "isExternalLink": true - }, - { - "cardItem": "What is an actor?", - "href": "/platform/actors" - }, - { - "cardItem": "What is web scraping?", - "href": "/academy/web-scraping-for-beginners" - } - ], - "Guides": [ - { - "cardItem": "Actor tasks", - "href": "/platform/actors/running/tasks" - }, - { - "cardItem": "Run an Actor and retrieve data via API", - "href": "/platform/tutorials/run-actor-and-retrieve-data-via-api" - }, - { - "cardItem": "Log into a website", - "href": "/academy/puppeteer-playwright/common-use-cases/logging-into-a-website" - }, - { - "cardItem": "Integrations", - "href": "/platform/integrations" - }, - { - "cardItem": "Use Apify from PHP", - "href": "/academy/php/using-apify-scraper-with-php" - }, - { - "cardItem": "Automate repetitive tasks", - "href": "/academy/concepts/robotic-process-automation" - }, - { - "cardItem": "Process data in Python with Pandas", - "href": "/academy/python/process-data-using-python" - } - ], - "Web scraping": [ - { - "cardItem": "Web scraping academy", - "href": "https://developers.apify.com/academy/web-scraping-for-beginners", - "isExternalLink": true - }, - { - "cardItem": "Basics of data collection", - "href": "https://developers.apify.com/academy/web-scraping-for-beginners/data-collection", - "isExternalLink": true - }, - { - "cardItem": "Scrape Twitter", - "href": "https://blog.apify.com/how-to-scrape-tweets-and-more-on-twitter-59330e6fb522", - "isExternalLink": true - }, - { - "cardItem": "Scrape Facebook", - "href": "https://blog.apify.com/how-to-scrape-facebook-pages-posts-comments-photos-and-more-425ebef352d8", - "isExternalLink": true - }, - { - "cardItem": "Scrape Amazon", - "href": "https://blog.apify.com/step-by-step-guide-to-scraping-amazon-c3c8b481333", - "isExternalLink": true - }, - { - "cardItem": "Analyze page content before scraping", - "href": "/academy/concepts" - }, - { - "cardItem": "Analyze pages and fix errors", - "href": "/academy/node-js/analyzing-pages-and-fixing-errors" - }, - { - "cardItem": "Scrape data in Python with Beautiful Soup", - "href": "/academy/python/scrape-data-python" - } - ], - "Platform features": [ - { - "cardItem": "Proxy", - "href": "/platform/proxy" - }, - { - "cardItem": "API Docs", - "href": "/api/v2" - }, - { - "cardItem": "Integrate with the Apify platform", - "href": "/platform/integrations" - }, - { - "cardItem": "Manage data", - "href": "/platform/storage" - }, - { - "cardItem": "Schedule actor runs", - "href": "/platform/schedules" - }, - { - "cardItem": "Monitor actor and task runs", - "href": "/platform/monitoring" - } - ], - "Reduce blocking": [ - { - "cardItem": "Connect to Apify Proxy", - "href": "/platform/proxy/connection-settings" - }, - { - "cardItem": "Bypass anti-scraping techniques", - "href": "/academy/anti-scraping/techniques" - }, - { - "cardItem": "Use a proxy in your browser", - "href": "https://help.apify.com/en/articles/3858935-how-to-use-a-proxy-in-your-browser", - "isExternalLink": true - }, - { - "cardItem": "Manage proxies in Apify SDK", - "href": "https://sdk.apify.com/docs/guides/proxy-management", - "isExternalLink": true - }, - { - "cardItem": "Reduce blocking using sessions in SDK", - "href": "https://sdk.apify.com/docs/guides/session-management", - "isExternalLink": true - } - ], - "Build actors": [ - { - "cardItem": "Meet the Apify SDK", - "href": "https://sdk.apify.com", - "isExternalLink": true - }, - { - "cardItem": "Meet Crawlee", - "href": "https://crawlee.dev/docs", - "isExternalLink": true - }, - { - "cardItem": "Build your first crawler", - "href": "https://crawlee.dev/docs/quick-start", - "isExternalLink": true - }, - { - "cardItem": "Apify SDK API reference", - "href": "https://sdk.apify.com/api/apify", - "isExternalLink": true - }, - { - "cardItem": "Actor code examples", - "href": "https://crawlee.dev/docs/examples/basic-crawler", - "isExternalLink": true - }, - { - "cardItem": "Crawl a site using Cheerio Scraper", - "href": "https://crawlee.dev/docs/examples/cheerio-crawler", - "isExternalLink": true - }, - { - "cardItem": "Manage requests in SDK", - "href": "https://sdk.apify.com/docs/guides/request-storage", - "isExternalLink": true - }, - { - "cardItem": "Manage results in SDK", - "href": "https://sdk.apify.com/docs/guides/result-storage", - "isExternalLink": true - } - ] -} +[ + { + "title": "Actors", + "description": "Learn how to develop, run and share own web scraping and automation tools.", + "to": "/platform/actors" + }, + { + "title": "Storage", + "description": "Store anything from images and key-value pairs to structured output data.", + "to": "/platform/storage" + }, + { + "title": "Proxy", + "description": "Learn to anonymously and reliably access websites in scraping/automation jobs.", + "to": "/platform/proxy" + }, + { + "title": "Schedules", + "description": "Learn how to automatically start your Actor and task runs.", + "to": "/platform/schedules" + }, + { + "title": "Integrations", + "description": "Learn how to connect the Apify platform with your projects.", + "to": "/platform/integrations" + }, + { + "title": "Monitoring", + "description": "Learn how to check the performance of your Actors, validate your data and receive alerts.", + "to": "/platform/monitoring" + } +] diff --git a/sources/platform/index.mdx b/sources/platform/index.mdx index d3358a3e55..fef4ae7dab 100644 --- a/sources/platform/index.mdx +++ b/sources/platform/index.mdx @@ -1,44 +1,52 @@ --- title: Home -description: Mold our tools any way you want to scrape websites or automate repetitive tasks. Find the solution to your task here or use the Search box above. -sidebar_position: 1 +description: Apify is your one-stop shop for web scraping, data extraction, and RPA. Automate anything you can do manually in a browser. slug: / hide_table_of_contents: true +sidebar_position: 0 --- -import PlatformCard from "@site/src/components/PlatformCard"; +import Card from "@site/src/components/Card"; import CardGrid from "@site/src/components/CardGrid"; import homepageContent from "./homepage_content.json"; -Mold our tools any way you want to scrape websites or automate repetitive tasks. Find the solution to your task here or use the Search box above. +# Apify Platform + +> **Apify** is a cloud platform that helps you build reliable web scrapers, fast, and automate anything you can do manually in a web browser. +> +> **Actors** are serverless cloud programs running on the Apify platform that can easily crawl websites with millions of pages, but also perform arbitrary computing jobs such as sending emails or data transformations. They can be started manually, using our API or scheduler, and they can be easily integrated with other apps. + +## Getting started + +**Learn how to run any Actor in Apify Store or create your own. A step-by-step guides through your first steps on the Apify platform.** + + + + + + + +## Contents { - Object.entries(homepageContent).map(([title, items]) => ( - ( + ) ) } - - - - diff --git a/sources/platform/schedules.md b/sources/platform/schedules.md index 21c2ec3bf4..79035677ae 100644 --- a/sources/platform/schedules.md +++ b/sources/platform/schedules.md @@ -151,4 +151,4 @@ Additionally, you can use the following shortcut expressions: ## Sharing -You can invite other Apify users to view or modify your schedules using the [access rights](./access_rights/index.md) system. See the [full list of permissions](./access_rights/list_of_permissions.md). +You can invite other Apify users to view or modify your schedules with the [access rights](./collaboration/index.md) system. See the [full list of permissions](./collaboration/list_of_permissions.md). diff --git a/sources/platform/storage/dataset.md b/sources/platform/storage/dataset.md index 54ccd64e5e..4e704ea404 100644 --- a/sources/platform/storage/dataset.md +++ b/sources/platform/storage/dataset.md @@ -40,7 +40,7 @@ Only named datasets are displayed by default. Select the **Include unnamed datas ![Datasets in app](./images/datasets-app.png) To view or download a dataset in the above-mentioned formats, click on its **Dataset ID**. Under the **Settings** tab, you can update the dataset's name (and, in turn, its [retention period](./index.md)) and -[access rights](../access_rights/index.md). Click on the `API` button to view and test the dataset's [API endpoints](/api/v2#/reference/datasets). +[access rights](../collaboration/index.md). Click on the `API` button to view and test the dataset's [API endpoints](/api/v2#/reference/datasets). ![Datasets detail view](./images/datasets-detail.png) @@ -321,7 +321,7 @@ By default, the whole result is wrapped in an `` element, while each pag ## Sharing {#sharing} -You can invite other Apify users to view or modify your datasets using the [access rights](../access_rights/index.md) system. See the [full list of permissions](../access_rights/list_of_permissions.md). +You can invite other Apify users to view or modify your datasets with the [access rights](../collaboration/index.md) system. See the [full list of permissions](../collaboration/list_of_permissions.md). ### Sharing datasets between runs {#sharing-datasets-between-runs} diff --git a/sources/platform/storage/index.md b/sources/platform/storage/index.md index 7455091bfc..41a2700002 100644 --- a/sources/platform/storage/index.md +++ b/sources/platform/storage/index.md @@ -94,7 +94,7 @@ To access your storages from Apify Console, go to the [**Storage** section](http You can edit your stores' names by clicking their caption (ID or name) on their detail page. -Under the **Settings** tab of their detail page, you can grant [access rights](../access_rights/index.md) to other Apify users. +Under the **Settings** tab of their detail page, you can grant [access rights](../collaboration/index.md) to other Apify users. You can quickly share your storages' contents and details by sharing the URLs you find under the **API** tab in a store's detail page. @@ -191,7 +191,7 @@ For example, the storage names **janedoe~my-storage-1** and **janedoe~web-scrape ## Sharing {#sharing} -You can invite other Apify users to view or modify your storages using the [access rights](../access_rights/index.md) system. See the [full list of permissions](../access_rights/list_of_permissions.md). +You can invite other Apify users to view or modify your storages with the [access rights](../collaboration/index.md) system. See the [full list of permissions](../collaboration/list_of_permissions.md). ### Sharing storages between runs {#sharing-storages-between-runs} diff --git a/sources/platform/storage/key_value_store.md b/sources/platform/storage/key_value_store.md index c2dd2c2670..c29739bf3c 100644 --- a/sources/platform/storage/key_value_store.md +++ b/sources/platform/storage/key_value_store.md @@ -40,7 +40,7 @@ Only named key-value stores are displayed by default. Select the **Include unnam ![Key-value stores in app](./images/key-value-stores-app.png) To view a key-value store's content, click on its **Store ID**. -Under the **Settings** tab, you can update the store's name (and, in turn, its [retention period](./index.md)) and [access rights](../access_rights/index.md). +Under the **Settings** tab, you can update the store's name (and, in turn, its [retention period](./index.md)) and [access rights](../collaboration/index.md). Click on the `API` button to view and test a store's [API endpoints](/api/v2#/reference/key-value-stores). ![Key-value stores detail](./images/key-value-stores-detail.png) @@ -204,7 +204,7 @@ You can compress a record and use the [Content-Encoding request header](https:// ## Sharing -You can invite other Apify users to view or modify your key-value stores using the [access rights](../access_rights/index.md) system. See the [full list of permissions](../access_rights/list_of_permissions.md). +You can invite other Apify users to view or modify your key-value stores with the [access rights](../collaboration/index.md) system. See the [full list of permissions](../collaboration/list_of_permissions.md). ### Sharing key-value stores between runs diff --git a/sources/platform/storage/request_queue.md b/sources/platform/storage/request_queue.md index 0a3e0bd1f5..dc5baa4563 100644 --- a/sources/platform/storage/request_queue.md +++ b/sources/platform/storage/request_queue.md @@ -39,7 +39,7 @@ Only named request queues are displayed by default. Select the **Include unnamed To view a request queue, click on its **Queue ID**. Under the **Settings** tab, you can update the queue's name (and, in turn, its -[retention period](./index.md)) and [access rights](../access_rights/index.md). +[retention period](./index.md)) and [access rights](../collaboration/index.md). Click on the `API` button to view and test a queue's [API endpoints](/api/v2#/reference/request-queues). ![Request queues detail](./images/request-queue-detail.png) @@ -217,7 +217,7 @@ See the [API documentation](/api/v2#/reference/request-queues) for a detailed br ## Sharing {#sharing} -You can invite other Apify users to view or modify your request queues using the [access rights](../access_rights/index.md) system. See the [full list of permissions](../access_rights/list_of_permissions.md). +You can invite other Apify users to view or modify your request queues with the [access rights](../collaboration/index.md) system. See the [full list of permissions](../collaboration/list_of_permissions.md). ### Sharing request queues between runs {#sharing-request-queues-between-runs} diff --git a/sources/platform/tutorials/images/actor-input-view.png b/sources/platform/tutorials/images/actor-input-view.png deleted file mode 100644 index cb5e154358..0000000000 Binary files a/sources/platform/tutorials/images/actor-input-view.png and /dev/null differ diff --git a/sources/platform/tutorials/images/actor-run-dataset.png b/sources/platform/tutorials/images/actor-run-dataset.png deleted file mode 100644 index 134eaed9fa..0000000000 Binary files a/sources/platform/tutorials/images/actor-run-dataset.png and /dev/null differ diff --git a/sources/platform/tutorials/images/actor-run-results.png b/sources/platform/tutorials/images/actor-run-results.png deleted file mode 100644 index e162b4087f..0000000000 Binary files a/sources/platform/tutorials/images/actor-run-results.png and /dev/null differ diff --git a/sources/platform/tutorials/images/actor-run-view.png b/sources/platform/tutorials/images/actor-run-view.png deleted file mode 100644 index 0398f44e51..0000000000 Binary files a/sources/platform/tutorials/images/actor-run-view.png and /dev/null differ diff --git a/sources/platform/tutorials/images/apify-store.png b/sources/platform/tutorials/images/apify-store.png deleted file mode 100644 index 22833affd4..0000000000 Binary files a/sources/platform/tutorials/images/apify-store.png and /dev/null differ diff --git a/sources/platform/tutorials/images/create-actor-set-input.png b/sources/platform/tutorials/images/create-actor-set-input.png deleted file mode 100644 index dc05e65c5c..0000000000 Binary files a/sources/platform/tutorials/images/create-actor-set-input.png and /dev/null differ diff --git a/sources/platform/tutorials/images/create-actor.png b/sources/platform/tutorials/images/create-actor.png deleted file mode 100644 index 390fb3d0d5..0000000000 Binary files a/sources/platform/tutorials/images/create-actor.png and /dev/null differ diff --git a/sources/platform/tutorials/images/edit-this-cookie.png b/sources/platform/tutorials/images/edit-this-cookie.png deleted file mode 100644 index 69caa3e5c1..0000000000 Binary files a/sources/platform/tutorials/images/edit-this-cookie.png and /dev/null differ diff --git a/sources/platform/tutorials/images/facebook-login.png b/sources/platform/tutorials/images/facebook-login.png deleted file mode 100644 index 13134784cf..0000000000 Binary files a/sources/platform/tutorials/images/facebook-login.png and /dev/null differ diff --git a/sources/platform/tutorials/images/link-remote-file.png b/sources/platform/tutorials/images/link-remote-file.png deleted file mode 100644 index 5006a4fc69..0000000000 Binary files a/sources/platform/tutorials/images/link-remote-file.png and /dev/null differ diff --git a/sources/platform/tutorials/images/make-link-viewable.png b/sources/platform/tutorials/images/make-link-viewable.png deleted file mode 100644 index 78c8eeb381..0000000000 Binary files a/sources/platform/tutorials/images/make-link-viewable.png and /dev/null differ diff --git a/sources/platform/tutorials/images/network-tab.png b/sources/platform/tutorials/images/network-tab.png deleted file mode 100644 index a2667f47db..0000000000 Binary files a/sources/platform/tutorials/images/network-tab.png and /dev/null differ diff --git a/sources/platform/tutorials/images/open-edit-this-cookie.png b/sources/platform/tutorials/images/open-edit-this-cookie.png deleted file mode 100644 index 340351ca19..0000000000 Binary files a/sources/platform/tutorials/images/open-edit-this-cookie.png and /dev/null differ diff --git a/sources/platform/tutorials/images/pagination-filters.png b/sources/platform/tutorials/images/pagination-filters.png deleted file mode 100644 index 22250c2a30..0000000000 Binary files a/sources/platform/tutorials/images/pagination-filters.png and /dev/null differ diff --git a/sources/platform/tutorials/images/pagination.png b/sources/platform/tutorials/images/pagination.png deleted file mode 100644 index e5833354d0..0000000000 Binary files a/sources/platform/tutorials/images/pagination.png and /dev/null differ diff --git a/sources/platform/tutorials/images/run-log-2.png b/sources/platform/tutorials/images/run-log-2.png deleted file mode 100644 index 89f9a1b90f..0000000000 Binary files a/sources/platform/tutorials/images/run-log-2.png and /dev/null differ diff --git a/sources/platform/tutorials/images/run-log.png b/sources/platform/tutorials/images/run-log.png deleted file mode 100644 index 62c4636f46..0000000000 Binary files a/sources/platform/tutorials/images/run-log.png and /dev/null differ diff --git a/sources/platform/tutorials/images/start-url.png b/sources/platform/tutorials/images/start-url.png deleted file mode 100644 index 8e597a7ccb..0000000000 Binary files a/sources/platform/tutorials/images/start-url.png and /dev/null differ diff --git a/sources/platform/tutorials/images/start-urls-in-spreadsheet.png b/sources/platform/tutorials/images/start-urls-in-spreadsheet.png deleted file mode 100644 index ed5e9f52bf..0000000000 Binary files a/sources/platform/tutorials/images/start-urls-in-spreadsheet.png and /dev/null differ diff --git a/sources/platform/tutorials/images/web-scraper-input.png b/sources/platform/tutorials/images/web-scraper-input.png deleted file mode 100644 index 46480e0da6..0000000000 Binary files a/sources/platform/tutorials/images/web-scraper-input.png and /dev/null differ diff --git a/sources/platform/tutorials/index.md b/sources/platform/tutorials/index.md deleted file mode 100644 index f7d1d1a0eb..0000000000 --- a/sources/platform/tutorials/index.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -title: Tutorials -description: Learn how to scrape the web and automate processes with Apify. From beginner guides for using actors to advanced topics like migrations and performance. -sidebar_position: 3 -category: guides -slug: /tutorials ---- - -# Tutorials - -**Learn how to scrape the web and automate processes with Apify. From beginner guides for using actors to advanced topics like migrations and performance.** - ---- - -This section contains everything you need to get you up and running with Apify, as well as advanced techniques for experienced users. Before you start, it will help to have an [Apify account](https://console.apify.com) set up and ready to go. - -## The basics - -[Quick start](./quick_start.md) will show you how to [run existing actors](./quick_start.md) and [create your first Apify actor](./quick_start.md#create-an-actor). - -[Set up your first web scraping job](/academy/apify-scrapers/getting-started) with a step-by-step guide. - -## Next steps - -If you liked what you learned, get familiar with our main tools by following -the [tutorial](/academy/apify-scrapers/web-scraper) for **Web Scraper** ([apify/web-scraper](https://apify.com/apify/web-scraper)), -the [tutorial](/academy/apify-scrapers/cheerio-scraper) for **Cheerio Scraper** ([apify/cheerio-scraper](https://apify.com/apify/cheerio-scraper)), -or the [tutorial](/academy/apify-scrapers/puppeteer-scraper) for **Puppeteer Scraper** ([apify/puppeteer-scraper](https://apify.com/apify/puppeteer-scraper)) - -Or, if the solution you need is beyond your skillset or schedule, you can always [order a custom solution](https://apify.com/custom-solutions). - -## Building actors - -Building your own actors requires some experience with coding. If you're ready, get started with the [Apify SDK](/sdk/js/) and [Crawlee](https://crawlee.dev/docs/quick-start). - -"But wait, I don't speak JavaScript", we hear you say. Worry not! While we currently only have an SDK for JavaScript, your actor can be in any programming language. If you use Python, you can now [scrape data with Beautiful Soup](/academy/python/scrape-data-python) -and [parse the results with Pandas](/academy/python/process-data-using-python) on the Apify platform. Prefer PHP? Cool! Here's how you can [use Apify in your PHP projects](/academy/php/using-apify-scraper-with-php). - -## Advanced use cases - -Check out our other tutorials that cover more specific and advanced scenarios such as [scraping dynamic content](/academy/puppeteer-playwright/page/waiting), [integrating](../integrations/index.md) your projects with Apify, [using your actors via API](./run_actor_and_retrieve_data_via_api.md), or setting up [monitoring](../monitoring/index.md) for your projects. - -## Finally - -We're constantly working on extending and improving this section. If you have suggestions or requests for specific tutorials, let us know using the form below. diff --git a/sources/platform/tutorials/quick_start.md b/sources/platform/tutorials/quick_start.md deleted file mode 100644 index d807662853..0000000000 --- a/sources/platform/tutorials/quick_start.md +++ /dev/null @@ -1,132 +0,0 @@ ---- -title: Quick start -description: Learn how to run any actor in Apify Store or create your own. A step-by-step guide through your first steps on the Apify platform. -sidebar_position: 3.1 -slug: /tutorials/quick-start ---- - -# Quick start - -**Learn how to run any actor in Apify Store or create your own. A step-by-step guide through your first steps on the Apify platform.** - ---- - -This article shows you the very basics of using the Apify platform. It shows you the principles of running any actor in Apify Store and the first steps of building an actor using the in-app code editor. - -**You will need an Apify account to complete this tutorial. If you don't have one, [complete the sign-up process](https://console.apify.com/sign-up) first.** - -## Run an actor - -**1.** Navigate to [Apify Store](https://console.apify.com/store) and pick any actor you like the look of. We'll pick the [Google Search Results Scraper](https://console.apify.com/actors/nFJndFXA5zjCTuudP#/information/latest/readme). - -![Apify store](./images/apify-store.png) - -**2.** On the actor's page, head over to the **Input** tab. - -Don't be put off by all the boxes - the actor is pre-configured to run without any extra input. Just click the **Start** button in the bottom-left corner. - -Alternatively, you can play around with the settings to make the results more interesting for you. - -![Actor input view](./images/actor-input-view.png) - -**3.** Wait for the actor to run... - -**4.** Get your results! - -Click on the flashing **Results** button. - -![Actor successful run view](./images/actor-run-view.png) - -Then, choose which format you want your data in and check it out. - -![Choose the data format](./images/actor-run-results.png) - -Here is a preview of the actor's output as a table. - -![Dataset preview](./images/actor-run-dataset.png) - -## Create an actor - -**1.** [Go to the **Actors** section](https://console.apify.com/actors) in the Apify Console and click on the **Create new** button. - -![Create a new actor](./images/create-actor.png) - -**2.** Go to the **Source** tab and paste the Node.js code below into the **Source code** editor. - -```js -import { Actor } from 'apify'; - -await Actor.init(); - -console.log('Hello world from Actor!'); - -await Actor.exit(); -``` - -Click **Save**. - -**3.** Click **Build** in the **Developer console** below to build your actor. After the build is finished, click **▷ Start** to run your actor. After the run is finished, you should see something like: - -![Apify actor run log](./images/run-log.png) - -Congratulations, you have successfully created and run your first actor! - -### Accepting input - -Let's try something little more advanced. We will enable the Actor to accept input and generate output (for more details, [see the Input article](../actors/running/input_and_output.md)). - -**1.** Paste the code below into the **Source code** editor. - -```js -import { Actor } from 'apify'; - -await Actor.init(); - -// Get input and print it -const input = await Actor.getInput(); -console.log('My input:'); -console.dir(input); - -await Actor.exit(); -``` - -**2.** Save your actor by clicking **Save** and then rebuild it by clicking **Build**. - -**3.** After the build is finished, go to the **Developer console** and set the **Input** to: - -```json -{ "hello": 123 } -``` - -![Apify actor run log](./images/create-actor-set-input.png) - -**4.** Click **▷ Start**. When the run finishes, you will see something like: - -![Apify actor run log](./images/run-log-2.png) - -Excellent, you have just created your first actor that accepts input and logs output! Now you can start adding some magic. - -You can program an Apify actor to do anything one can do in the browser, from sending emails to processing vast amounts of data. For examples and inspiration, [visit Apify Store](https://apify.com/store). - -### Building more advanced actors - -The above actor (and many others) uses the `apify` [NPM package](https://www.npmjs.com/package/apify), which simplifies the development of actors. The Apify SDK has tutorials on how you can [get started](/sdk/js/docs/guides/getting-started). Also make sure to check out [`Crawlee`](https://crawlee.dev/) - the library containing crawling and scraping-related tools, which previously were a part of Apify SDK. - -If you are building your own actors, you'll probably prefer to host the source code on Git. To do that, follow these steps: - - - -1. Create a new Git repository. -2. Copy the boilerplate actor code from the [apify/quick-start](https://github.com/apify/actor-quick-start) actor. -3. Set **Source type** to **Git repository** for your actor in the app. -4. Paste the Git repo link to **Git URL**, save changes and build your actor. -5. That's it, now you can develop your actor locally on your computer and run it in the Apify cloud! - -For more information, see the [Git repository](../actors/development/source_code.md) section. - -## What's next? - -* Try some other Actors from [Apify Store](https://apify.com/store). -* Learn to use our main scrapers: [Web Scraper](/academy/apify-scrapers/web-scraper), [Cheerio Scraper](/academy/apify-scrapers/cheerio-scraper), and [Puppeteer Scraper](/academy/apify-scrapers/puppeteer-scraper). -* Learn more about using Actors and the Apify platform: [tasks](../actors/running/tasks.md), [storages](../storage/index.md), and [proxies](../proxy/index.md). -* Set up the [`apify`](/sdk/js/docs/guides/getting-started) and [`crawlee`](https://crawlee.dev/docs/quick-start) packages on your computer, and build your own actors with the Apify SDK and Crawlee. diff --git a/src/components/AcademyCard.jsx b/src/components/Card.jsx similarity index 82% rename from src/components/AcademyCard.jsx rename to src/components/Card.jsx index ec8d123a46..0ab434b310 100644 --- a/src/components/AcademyCard.jsx +++ b/src/components/Card.jsx @@ -7,9 +7,7 @@ const AcademyCard = ({ to, imageUrl, title, desc }) => { return (
-
- -
+ {!imageUrl || (
)}

{title}

{desc}