Learn

Tutorials, training, courses, and educational resources 2113 public listings.

• What is a web crawler used for? (ScrapingBee)
  A web crawler is a "bot" generally used by search engines to look for new websites, download their data, and index it.
• How to fix ReadTimeout error in Python requests? (ScrapingBee)
  ReadTimeout occurs when the website you are trying to connect to doesn't send back data in time.
• How to remove any element from the HTML response (ScrapingBee)
  Sometimes you may need to remove specific HTML elements from the page's content to get cleaner results for your data extraction rules, or t…
• How to download a file with Playwright and Python? (ScrapingBee)
  You can download a file with Playwright by targeting the file download button on the page using any Locator and clicking it.
• How to turn HTML to text in Python? (ScrapingBee)
  You can easily extract text from an HTML page using any of the famous HTML parsing libraries in Python.
• How to make screenshots in C# (ScrapingBee)
  Taking a screenshot of your website is very straightforward using ScrapingBee. You can either take a screenshot of the visible portion of t…
• How to select elements by text in XPath? (ScrapingBee)
  Learn how to use XPath by text with exact and partial matches. See how XPath text(), contains(), and equals work with practical examples in…
• How to handle infinite scroll pages in PHP (ScrapingBee)
  Many websites use infinite scrolling to load content as you scroll down the page instead of loading everything at once.
• Scrolling and loading more content via a JS scenario (ScrapingBee)
  Certain websites may require you to scroll in order to load more results on the page or within a specific element.
• What is requests used for in Python? (ScrapingBee)
  Requests is an HTTP library for Python-based programs. It is one of the most downloaded Python packages.
• How to scrape tables with Cheerio? (ScrapingBee)
  You can scrape tables with Cheerio by combining the regular CSS selectors with the find and each methods to iterate over the rows and cells…
• How to take a screenshot with Selenium? (ScrapingBee)
  You can take a screenshot using the selenium web driver via the save_screenshot method.
• How to extract a table (ScrapingBee)
  Data can be found online in various formats, but the most popular one is table format, especially that it displays information in a very st…
• How to find HTML element by class with BeautifulSoup? (ScrapingBee)
  To extract HTML elements with a specific class name using BeautifulSoup, we use the attrs parameter of the functions find or find_all.
• How to handle infinite scroll pages in Go (ScrapingBee)
  Nowadays, most websites use different methods and techniques to decrease the load and data served to their clients’ devices.
• How to take a screenshot with Playwright? (ScrapingBee)
  Take a screenshot in Playwright with page.screenshot(), use the path argument to choose where the image is saved, and pass full_page=True f…
• How to follow redirect using cURL? (ScrapingBee)
  To follow redirect using cURL you need to use the -L option. Here is a sample command that sends a GET request to our hosted version of HTT…
• How to find all links using BeautifulSoup and Python? (ScrapingBee)
  You can find all of the links, anchor elements, on a web page by using the find_all function of Beautiful Soup4, with the tag "a" as a para…
• How to find elements by XPath selectors in Playwright? (ScrapingBee)
  Use XPath in Playwright by passing the selector to page.locator() with an xpath= prefix or a leading //, then read text, click, or fill the…
• Adding items to an eCommerce shopping cart (ScrapingBee)
  Here is a quick tutorial on how you may add items to a shopping cart on eCommerce websites using ScrapingBee API via a JS scenario on Pytho…
• How to Wait for Page to Load in Playwright? (ScrapingBee)
  Wait for the specific page content you need with page.wait_for_selector() or Playwright's built-in auto-waiting.
• How to save and load cookies in Puppeteer? (ScrapingBee)
  Learn how to save and load cookies in Puppeteer using page.cookies() and page.setCookie(), with a practical JavaScript example.
• How to block resources in Playwright and Python? (ScrapingBee)
  You can block resources in Playwright by making use of the route method of the Page or Browser object and registering an interceptor that r…
• How to handle infinite scroll pages in Python (ScrapingBee)
  Nowadays, most websites use different methods and techniques to decrease the load and data served to their clients’ devices.
• How to POST JSON Using cURL? (ScrapingBee)
  Use the cURL --json option to send a JSON POST request.
• How to find elements by CSS selector in Selenium? (ScrapingBee)
  Find elements by CSS selector in Selenium with find_element() and find_elements(), using By.CSS_SELECTOR to match by tag, class, ID, attrib…
• 503 status code - what is it and how to avoid it? (ScrapingBee)
  Learn what a 503 Service Unavailable status code means, common causes, and how to avoid 503 errors when scraping websites.
• How to run Playwright in Jupyter notebooks? (ScrapingBee)
  You can run Playwright in Jupyter notebooks by making use of Playwright's async API. This is required because Jupyter notebooks use an asyn…
• How to get XML with cURL ? (ScrapingBee)
  Learn how to get XML with cURL using GET requests and Accept headers, with simple command-line examples for XML responses.
• How to extract curl requests from Safari (ScrapingBee)
  This tutorial from scrapingbee.com covers How to extract curl requests from Safari as a standalone official training item (Ready to get sta…
• How to find all links using DOM Crawler and PHP? (ScrapingBee)
  Extracting links from HTML in PHP is simple with DOM Crawler. This guide shows how to find all links on a page, filter them, and grab their…
• How to use CSS Selectors in Python? (ScrapingBee)
  CSS selectors are patterns that are used to reference HTML elements, primarily for the purpose of styling them using CSS.
• How to get page source in Selenium? (ScrapingBee)
  You can easily get the page source in Selenium via the page_source attribute of the Selenium web driver.
• What does Beautifulsoup do in Python? (ScrapingBee)
  Learn what BeautifulSoup does in Python, how it parses HTML, and how to use it to extract data from web pages.
• 429 status code - what is it and how to avoid it? (ScrapingBee)
  Learn what a 429 Too Many Requests status code means, why rate limits happen, and how to avoid 429 errors when scraping.
• Getting started with ScrapingBee and Ruby (ScrapingBee)
  In this tutorial, we will see how you can use Scraping Bee’s API with Ruby, and use it to scrape web pages.
• How to find elements by XPath in Selenium? (ScrapingBee)
  Find elements by XPath in Selenium with find_element(By.XPATH, ...) for the first match or find_elements() for every match, plus XPath synt…
• How to make screenshots in PHP (ScrapingBee)
  You can take a website screenshot in PHP by sending a cURL request to the ScrapingBee API.
• How to handle infinite scroll pages in C# (ScrapingBee)
  Nowadays, most websites use different methods and techniques to decrease the load and data served to their clients’ devices.
• 403 status code - what is it and how to avoid it? (ScrapingBee)
  Learn what a 403 Forbidden status code means, why scraping requests get blocked, and how to avoid 403 errors responsibly.
• How long does it take to learn web scraping in Python? (ScrapingBee)
  Depending on your Python knowledge, and how much time you're allocating to learn this skill, it could take anywhere from two days to two ye…
• How to ignore invalid and self-signed certificates using cURL? (ScrapingBee)
  To ignore invalid and self-signed certificates using cURL you need to use the -k option. Here is a sample command that sends a GET request…
• What is Puppeteer? (ScrapingBee)
  Puppeteer is a browser automation library developed by the Chrome Dev Tools team. Simply put, it is a tool that allows you to control your…
• Getting Started with ScrapingBee (ScrapingBee)
  If you’re looking for a web scraping API for Node.js, you can use the ScrapingBee SDK, which acts as a wrapper around ScrapingBee’s API.
• How to scrape tables with BeautifulSoup? (ScrapingBee)
  We can parse a table's content with Beautiful Soup by finding all elements, and finding their or children.
• How to get JSON with cURL ? (ScrapingBee)
  You can get JSON with cURL by sending a GET request with the -H "Accept: application/json" option.
• How to find HTML elements by class? (ScrapingBee)
  You can find HTML elements by class via multiple ways in Python. The method you choose will depend on the library you are using.
• How to extract curl requests from Chrome (ScrapingBee)
  Every time you perform an action on a web page, such as clicking a button, submitting a form, or loading another page, Chrome sends an HTTP…
• How to make screenshots in Go (ScrapingBee)
  Taking a screenshot of your website is very straightforward using ScrapingBee. You can either take a screenshot of the visible portion of t…