Often with many websites the data to be scraped may not be located at the same position within all pages, but is guaranteed to be found under a given heading (Example : “Technical Details”, “Product Specification” etc). Sometimes, the text under a given heading may not be selected as a single item during configuring. In such scenarios the ‘Capture following text’ option in the capture window will provide helpful.

How to ?

While in configuration mode, click on the heading and select the ‘Capture following text’ option in the capture window. Provide a suitable name for the field and hit OK. In the preview pane you will be able to see the text following the heading captured.

Refer http://www.webharvy.com/tour1.html#ScrapeFollowingText for more details.

• New option: ‘Capture following text’ added in capture form.
• Web Miner has been improved to handle even HTML errors of target websites.
• Allows exporting scraped data while mining is paused.
• For CSV, TSV exports, column names are added as the first row.
• Option to input keywords in CSV format.
• Option to manually set page load timeout value in application settings.

The ‘Capture following text’ feature helps to scrape text following a given heading within the page. This feature is useful when data to be scraped does not occur at a fixed position within the page, but is guaranteed to follow a heading text (Example ‘Product Details:‘ or ‘Specification‘).

The option to manually set the page load timeout value from settings window helps to scrape data from websites with slow response times or from those which employ AJAX.

We recommend that you download and try the 15 days free evaluation version.

• Added Test and Measurement (TMC) device class decoding
• Performance issues related to large data capture ( > 1GB) fixed
• Capture driver modifications : Performance improvements, bug fixes related to x64 version
• Added DEVICE_QUALIFIER_DESCRIPTOR decoding
• Decoding of String descriptor references in Device/Configuration/Interface descriptors
• Displays ‘descriptor type’ for GET/SET Descriptor requests in log view
• Modified HID class decoder to correctly open the selected device using the device serial number
• Application exit delay when log view is loaded with large size of (~ 1GB) captured data has been minimized
• Export utility has been modified to export captured data in a thread. Added Cancel option

Download : http://www.sysnucleus.com/usbtrace_download.html

WebHarvy provides you the option to specify either a single proxy server address or a list of proxy servers addresses through which the remote website will be scraped. In case you are providing a list of proxy server addresses, WebHarvy will cycle through the list in a periodic manner.

Please follow this link to know more about this feature.

Download WebHarvy Web Scraper FREE Trial !

The ‘Keyword Scraping’ feature of WebHarvy allows you to perform this task with ease. You can specify a list of input keywords and WebHarvy will automatically scrape data from the search results corresponding to each keyword in the specified list.

Please follow this link to know more about ‘Keyword based Scraping’.

Video Demonstration : Keyword based Scraping

We recommend that you download and try the evaluation version of our Web Scraper to know more about the features.

Please follow this link to know more about Category Scraping.

Category Scraping : Video demonstration 

You may download and try the free evaluation version of WebHarvy, the visual Web Scraper software, from http://www.webharvy.com/download.html.

Changes :

• [New Feature] Keyword based Scraping : Allows you to run the same configuration for a set of input keywords (Read more : http://www.webharvy.com/tour71.html)
• Edit Configuration : Allows you to edit an already saved WebHarvy configuration XML file (Read more : http://www.webharvy.com/tour41.html)
• Option to contact us (WebHarvy Support) directly from the application (See Help menu)
• Option to check for new updates directly from the application (See Help menu)
• Miner performance improvement : Web mining performance while following links from the main page has been improved
• Minor improvements and bug fixes
• Miner window remembers its last position/size/state
• Issue with Auto Scroll fixed
• Issue with loading ‘Next Page’ and ‘Following Links’ in certain scenarios while mining has been fixed
• Issue which resulted in application crash while parsing HTML of certain websites has been fixed

You may download the 15 days evaluation copy of WebHarvy Web Scraper from http://www.webharvy.com/download.html .

• Ability to append the mined data to already existing CSV, XML, TSV files without overwriting them.
• Option to ‘copy’ data directly from ‘Captured Data Preview’ pane so that it can be pasted to an excel (or any other spreadsheet) document.

To know more, please visit the Bluetooth decoder details page .