Have something to share about Test Automation or WebDriver?
Become a speaker now

Oracles in real time: data visualization on the command line

February 26th


Stage A


50 minutes


Test Oracles address a mutex problem that is faced by all teams who *successfully* practice continuous deployment. The problem test oracles address is: once your tests have run, and you have the results, how do you know what that means? In other words, what are you testing? What do your tests tell you? How do you know what you know?

Command line tools provide powerful data analytics when applied to a mature CI system and deployment pipeline. Such systems are known to be opaque — but the GNU/bash command line environment is powerful enough to solve those problems (with a little help from HTML5!).

A healthy CI system and deployment pipeline will naturally accrue a wealth of filesystem artifacts which can be mined for Test Oracles using command-line tools. Topics covered in depth will include:

• JSON testing and transformation with jq.
• Jenkins monitoring tips.
• Network analysis tools for Selenium developers.
• Visualizing xUnit test runner output.
• Analyzing your captured screenshots.
• Tips on introducing new dashboards to your team.

By providing yourself and your team not just with data but with actionable oracles, you will discover new reservoirs of activity that can safely become automated-rather-than-manual work.

Noah Sussman

Noah Sussman

Teachers Pay Teachers, USA

Noah Sussman is an automation engineer who studies how people and computers relate to each other. His approach to scaling Web sites, in particular CI systems, is demonstrably successful — he is noted for designing innovative test architectures for The SAT Test, Etsy and Barnes & Noble Nook. After several years of consulting, Noah has now joined the team at Teachers Pay Teachers – hello we are hiring engineers!