What is scraping integration and how does it work?
The scraping integration gives the possibility to automatically import current links of a website. For example, currently published articles from a news site enter the feed. The scraping integration primarily reads the URL of the website for new articles. For this, it needs a link as an example of what this URL and the articles found in it look like.
Example:
The URL that is read:
https://www.tagesschau.de/
The link template:
https://www.tagesschau.de/ausland/asien/gaza-israel-angriff-116.html
In this example, the scraping integration would read the basic structure of the URL and check for any new articles that are published.
In the example, the URL, i.e. the basic structure is: tagesschau.de/.
If a new article is published, the URL remains the same, only the link becomes longer with regard to the added article and this is then read by the integration and thus runs into the feed. Under the function "Load older cards" you can also upload past posts and content.
Example:
The URL that is read:
https://www.tagesschau.de/
The link template:
https://www.tagesschau.de/ausland/asien/gaza-israel-angriff-116.html
In this example, the scraping integration would read the basic structure of the URL and check for any new articles that are published.
In the example, the URL, i.e. the basic structure is: tagesschau.de/.
If a new article is published, the URL remains the same, only the link becomes longer with regard to the added article and this is then read by the integration and thus runs into the feed. Under the function "Load older cards" you can also upload past posts and content.
Updated on: 23/10/2023
Thank you!