Scrape Websites using PhantomJS and CasperJS

Enhance your JavaScript skills with our comprehensive course on front-end testing and web scraping using CasperJS, lodash, and jQuery.

  • Overview
  • Curriculum
  • Instructor
  • Review

Brief Summary

This course is all about becoming a pro at web scraping using JavaScript, CasperJS, and PhantomJS. You’ll tackle 5 projects to learn how to extract data and manipulate web pages like a boss. Perfect for anyone wanting to up their web game!

Key Points

  • Learn to scrape data from web pages using CasperJS.
  • Work through 5 hands-on projects to solidify concepts.
  • Understand advanced web scraping techniques.
  • Use PhantomJS and CasperJS for testing and UI interaction.
  • Capture, download, and save website data effectively.

Learning Outcomes

  • Gain confidence in web scraping with practical projects.
  • Master the use of CasperJS and PhantomJS for various tasks.
  • Be able to scrape dynamically loaded content.
  • Learn how to handle AJAX-loaded data.
  • Understand data management by saving results as tables or JSON.

About This Course

Become a better JavaScript Developer and Learn Front-End Testing. We'll use javascript, lodash and jquery for scraping.

In this course you will learn how to scrape data from web pages using CasperJS.

This course consists of 5 example projects to help you fully understand the powers of the headless browser using the CasperJS API.

What You Will Learn

You will gain a thorough understanding of advanced web scraping concepts and also gain an insight into how to use the CasperJS for Testing DOM manipulation and UI interaction.


What to Expect

  • We'll begin with an overview of how both PhantomJS and CasperJS works along with how to install these frameworks.

  • Next, we'll discuss what our workflow will look like and the options we can pass into a Casper object.

  • Then we'll dive into the meat of this course by working through 5 projects.


The Projects Will Cover

  • How to scrape websites that are rendered with JavaScript instead of standard HTML

  • How to wait for AJAX loaded data to appear before scraping elements

  • How to submit forms both for Authorization and when making searches

  • How to define navigation Steps - like logging into a site, clicking a button and following links

  • How to write and save specified data in tables then output as an .html file or as JSON.

  • And how to take screenshots both of full web pages and specific containers


What is PhantomJS?

PhantomJS is a Full Web Stack that employs a headless browser. Phantom gives us the power to perform many interesting actions on a web page, such as: performing page manipulation, simulating user interaction and the ability to dynamically capture and save website data.


What is CasperJS?

CasperJS is a stand-alone framework built on top Phantom and is compatible with most operating systems. The focus of this course will be on the Casper API and we'll be using this API to write all our web scraping scripts.


What You Should Know

You should already know JavaScript basics. Including what a callback function is. It will help if you know some jQuery. We use lodash in of our examples but only as a replacement for the built-in Map method that's part of the native Javascript API.

  • Know how to use JavaScript for Data Mining

  • Be able to Capture, Download and Save Website Data

  • Understand how to use CasperJS and PhantomJS

Instructor

Profile photo of Patrick Schroeder
Patrick Schroeder

Software engineer and entrepreneur.  I specialize in full-stack javascript development.  My favorite technologies are Angular and Node.js.  I'm currently working on serverless apps using Firebase Firestore and Google cloud functions running on Node.js.  I enjoy creating high quality courses that clearly explain complex topics.  All courses are captured in 1080p high resolution.  Please message me if you have any questions...

Review
4.9 course rating
4K ratings
ui-avatar of Mohammed AlJazaeri
Mohammed A.
2.0
4 years ago

Course for beginners
one big issue I found I doesn't use windows

  • Helpful
  • Not helpful
ui-avatar of Marcelo Leite Junior
Marcelo L. J.
4.5
5 years ago

Very simple and precise... I like the "small projects" lesson type. But still need more content.

  • Helpful
  • Not helpful
ui-avatar of Melikşah TANK
Melikşah T.
2.0
5 years ago

Anlatım detaylandırılmamış. Bazı şeyleri izleyen kişi nasıl olsa biliyorsa diyerek geçiştirerek anlatmış.

  • Helpful
  • Not helpful
ui-avatar of Alexander Hosseini
Alexander H.
1.0
6 years ago

I'm lost

  • Helpful
  • Not helpful
ui-avatar of Stefan Zima
Stefan Z.
2.5
6 years ago

Outdated

  • Helpful
  • Not helpful
ui-avatar of Ruan Osório
Ruan O.
1.0
6 years ago

Este curso fraco, longe do esperado. Alguns dos códigos-fonte postados ao lado dos vídeos não funcionam, pois os websites passaram por atualizações.

Didática do instrutor é bem ruim, pois os casos de uso utilizados para a exploração da ferramenta são maçantes o que torna o curso fraco.

  • Helpful
  • Not helpful
ui-avatar of Vhsharo
Vhsharo
3.5
6 years ago

I need the Mac OS steps. This course only has the Windows steps

  • Helpful
  • Not helpful
ui-avatar of Dave Kidd
Dave K.
3.0
6 years ago

When adding the path to phantomjs executable file, it would have been nice to have known that each path needs to be separated with a semi colon. Also, the command touch doesn't work on my machine. Also, when installed the node package, most people probably wouldn't have node installed, luckily I did and know how to install node, but assuming that they'll be able to use npm is going to confuse a lot of users. The main issue I'm seeing is things aren't explained in as much detail as they need to be.

  • Helpful
  • Not helpful
ui-avatar of Karl Clinger
Karl C.
5.0
7 years ago

I got what I wanted from this course.

  • Helpful
  • Not helpful
ui-avatar of Antonio Penta
Antonio P.
3.5
7 years ago

Basic course for using CasperJS, just some APIs are presented. It is more about CasperJS not much about PhantomJS
The examples are quite useful.

  • Helpful
  • Not helpful
Leave A Reply

Your email address will not be published. Required fields are marked *

Ratings

Courses You May Like

Lorem ipsum dolor sit amet elit
Show More Courses