Webscraping and RPA
Click
Description
Click Node does a mouse (double)click on a given coordinates.
Parameters
Two parameters, a pair of coordinates count in pixels from upper left corner, are required:
Parameter |
Type |
Description |
|---|---|---|
X (required) |
integer |
A horizontal coordinate count in pixels from left edge of the display. |
Y (required) |
integer |
A vertical coordinate count in pixels from the upper edge of the display. |
Double click |
bool (checkbox) |
If checked (True): mouse will perform a double-click, else single-click |
Click Image
Description
Finds provided image on currently scraped page and clicks at point that has coordinates (image_pos_X + X, image_pos_Y + Y). In other words icon finds the coordinates of image on page and clicks at point (X, Y), relatively to found image.
Parameters
Parameter |
Type |
Description |
|---|---|---|
Image path (required) |
string |
Path to image |
Click offset X (required) |
int/float |
Relative to image offset on axis X |
Click offset Y (required) |
int/float |
Relative to image offset on axis Y |
Wait
Description
Wait Node creates a pauses between pipeline steps. It waits (and does nothing, obviously) for a specified amount of ms and then lets the next step in a pipeline to proceed.
Parameters
Two parameters are required:
Parameter |
Type |
Description |
|---|---|---|
Miliseconds (required) |
int/float |
Waiting time interval in miliseconds, e.g. entry: 1000 → 1000 ms (1 s) waiting time before another step. |
Add random ms |
int/float |
Adds a random real picked from a uniform distribution defined on the interval (- entered value, + entered value). |
Write
Description
Types the text given via the entry box as if it were typed on the regular keyboard.
Parameters
Parameter |
Type |
Description |
|---|---|---|
Text (required) |
string |
The text to be typed by the write icon |
Use Key
Description
Hits a single key or a combination of keys on a keyboard. At least one key must be entered and a combination (series) of up to three keys can be set up.
Parameters
Parameter |
Type |
Description |
|---|---|---|
Key 1 (required) |
comboentry value |
The keyboard key to be clicked or, if other keys entered, the first key of a combination. Can be either selected from the combobox or written in manually. |
Key 2 |
comboentry value |
The second key of the combination. Can be either selected from the combobox or written in manually. |
Key 2 |
comboentry value |
The third key of the combination. Can be either selected from the combobox or written in manually. |
Open Browser
Description
Opens the system default web browser (for further usage such as Load Website etc.). Allows to choose driver to use.
For now available: Firefox (Geckodriver), Chrome.
Note: To use a driver an appropriate web browser should be installed. Driver itself will be installed automatically.
Parameters
Parameter |
Type |
Description |
|---|---|---|
Show browser |
boolean |
Show browser GUI |
Driver |
driver (combobox) |
Driver type (Firefox, Chrome) |
Load Website
Description
Loads website with a given URL.
Parameters
One parameter is required:
Parameter |
Type |
Description |
|---|---|---|
URL (required) |
string |
URL of the loaded website, e.g. https://forloop.ai/. |
Take screenshot |
boolean |
If screenshot is needed (e.g. for using in Browser View) |
Refresh Page Source
Description
Refreshes source code of the loaded website.
Click XPath Element
Description
Clicks on the website element specified by its XPath.
Parameters
Parameter |
Type |
Description |
|---|---|---|
XPath (required) |
string |
The XPath of the element to be clicked upon, e.g. //div[@class=”media-select__input-content”]//button. |
Scan Web Page
Description
Searches for different elements on page and highlights them in Browser View.
Available types of elements:
Tables
Bullet lists
Texts
Headlines
Links
Images
+ elements with custom XPath
Parameters
Parameter |
Type |
Description |
|---|---|---|
Tables |
boolean |
Include tables in search |
Bullet lists |
boolean |
Include bullet lists in search |
Texts |
boolean |
Include texts in search |
Headlines |
boolean |
Include headlines lists in search |
Links |
boolean |
Include links in search |
Images |
boolean |
Include images in search |
Custom XPath (optional) |
string |
Include elements with custom XPath |
Extract XPath Element
Description
Finds HTML element using XPath and exports it.
Also allows to load XPath of element from Browser View to the first parameter (XPath).
Parameters
Parameter |
Type |
Description |
|---|---|---|
XPath (required) |
string |
XPath of element to be extracted |
Output variable (required) |
string |
Name of variable to be created inside platform |
Output filename (required) |
string |
Name of file to be created |
Write in file mode |
string (combobox) |
|
Extract Multiple XPaths
Description
Finds multiple HTML elements using XPath and exports them.
Parameters
Parameter |
Type |
Description |
|---|---|---|
Extraction setup file path (required) |
string |
Path to .txt file with XPaths (1 per row) |
Output variable (required) |
string |
Name of variable to be created inside platform |
Extract Table XPath Element
Description
Finds table using XPath and exports it.
Also allows to load XPath of element from Browser View to the first parameter (XPath).
Parameters
Parameter |
Type |
Description |
|---|---|---|
XPath Rows (required) |
string |
XPath of table row |
XPath Columns (required) |
string |
XPath of table column |
Output variable (required) |
string |
Name of variable to be created inside platform |
Set Proxy
Description
Set’s a new proxy connection.
Parameters
Parameter |
Type |
Description |
|---|---|---|
IP (required) |
string |
IP of the proxy server. |
Port (required) |
integer |
Proxy server port. |
Username |
string |
Login username. |
Password |
string |
Login password. |
Click Name Element
Description
Clicks on the website element containing the given text.
Parameters
Parameter |
Type |
Description |
|---|---|---|
Text (required) |
string |
The text to be searched on the website and clicked upon. |
Click Id Element
Description
Clicks on the website element specified by its ID.
Parameters
Parameter |
Type |
Description |
|---|---|---|
Id |
string |
The ID of the element to be clicked upon. |
Get Current URL
Description
Saves the URL of web page, which is being currently scraped, to new variable inside platform.
Parameters
Parameter |
Type |
Description |
|---|---|---|
Output variable (required) |
string |
Name of variable to be created inside platform |
Close Browser
Description
Closes browser after all scraping processes are done.
Wait Until Element Is Located
Description
Creates a pause until a certain element is located. Can be useful for instance when dealing with popup windows on page.
Parameters
Parameter |
Type |
Description |
|---|---|---|
XPath (required) |
string |
XPath of element that should appear on page |
Extract Page Source
Description
Extracts the HTML source of currently scraped web page and saves it to new variable inside platform.
Parameters
Parameter |
Type |
Description |
|---|---|---|
Output variable (required) |
string |
Name of variable to be created inside platform |
Scroll Web Page
Description
Scrolls web page by X pixels in directions Up/Down.
Parameters
Parameter |
Type |
Description |
|---|---|---|
Scroll to |
string (combobox) |
The direction of scrolling (Up/Down) |
Scroll by (required) |
int |
Length of scrolling in pixels |
Scroll max |
boolean |
Scroll to the top / end of page |
Create Folder
Description
Creates a new folder with a given name at a given location.
Parameters
Two parameters, a name of the new folder and its location are required.
Parameter |
Type |
Description |
|---|---|---|
Folder location (req.) |
string |
Path to a directory in which the new folder will be created, eg. /Users/MrForloop/new_dir_place. |
Folder name (req.) |
string |
A name of the new folder, eg. my_new_dir. The final path to the new directory will be in the example case /Users/MrForloop/new_dir_place/my_new_dir. |
Move File
Description
Moves a file from it’s current directory to a different one.
Parameters
Parameter |
Type |
Description |
|---|---|---|
File (req.) |
string |
Path to a file to be moved, eg. /Users/awesome_file.flpl. |
Move to (req.) |
string |
Path to a directory to which the file will be moved into, eg. /Users/MrForloop. The final path to the moved file will be in the example case /Users/MrForloop/awesome_file.flpl. |
Create File Queue
Description
Creates a file queue folder whit a specified template suffix, i.e. .txt. Files in this queue can be further processed directly in Forloop.
Parameters
Parameter |
Type |
Description |
|---|---|---|
Root Folder (req.) |
string |
Path to a root folder where the queue directory will be created, eg. /Users/MrForloop/Desktop/. |
Queue name (req.) |
string |
A name of the to be created queue directory. |
Suffix |
combobox option |
Only files with this specific suffix in the queue folder will be processed via Process Item In Queue. |
Process Item In Queue
Description
Processes a next-in-the-line file in the file queue.
Parameters
Parameter |
Type |
Description |
|---|---|---|
New variable name (req.) |
string |
Name of the variable storing the data from the processed file. |
Delete File
Description
Deletes a specified file as if by using rm <path to the file> command on UNIX systems
or del <path to the file> on Win/DOS).
Parameters
Parameter |
Type |
Description |
|---|---|---|
File name (req.) |
string |
Path to the file, i.e. /Users/Delete/This/file.txt. |
Download Image
Description
Downloads image from URL.
Parameters
Parameter |
Type |
Description |
|---|---|---|
Image URL (required) |
string |
URL with image |
Output filename (required) |
string |
Name of image to be saved |
Download Images XPath
Description
Downloads image using XPath. Can be used for downloading multiple images at once.
Parameters
Parameter |
Type |
Description |
|---|---|---|
XPath (required) |
string |
XPath of image element |
Output filename (required) |
string |
Name of image to be saved. In case of downloading multiple images will be used as a directory name. |