Archive for January, 2018

What is your first impression of a word Serenity? We also weren’t sure and thus checked the Urban dictionary. What we have found was the exact description of our test framework, which we have been using for 1 year already.

“Serenity is a loving, funny and caring person. When she’s in love, she loves hard. She will take you back even if you hurt me. She puts people over herself. Hardworking and smart. She doesn’t cry but when she does it’s streams create oceans. She is loud and will do her best to make you the best person possible. She sees beauty in everything but herself. But she’s amazing person to be in a relationship with. She’ll give you her mind, body and soul even when you don’t deserve it. Irreplaceable and hard to come by. Don’t give up on her because she won’t give up on you. She finds light in the darkest things. She’s the one you want to take home to your family. Athletic and always smiling. Serenity is not a common name and not a common person, so hold on to her.”
“Serenity is the whole package”
“Damn she looks good”

In my work, I would like to share with you our ups and downs, why we select Serenity and why sometimes we hate it but still love it. Why pure Selenium and BDD are cool and how Serenity can beats Selenide.

Live with me all the major periods of automation development in the Fashionette. Why is Serenity cool but not really?

Layout testing seemed always a complex task. Galen Framework offers a simple solution: test location of objects relatively to each other on page. Using a special syntax and comprehensive rules you can describe any layout you can imagine. In this talk we will be talking about designing, implementing and running test cases with Galen Framework.

Everyone involved in the automated testing has to examine the autotest results. The bigger the project is, the more results one has to review. The world keeps getting faster, some products release several times a week if not a day.

Allure Framework is a popular instrument that makes analysis of autotest reports easier. This talk will cover its new version Allure 2. One of its main advantages is extendability: now you can adapt Allure to suite your needs using a plugin system. Besides that, it has a lot of interesting features ranging from test environment and test retries to text fixtures.

FaaS services (like AWS Lambda) allows flexibly run your separate functions with super-fast scaling, and great independency. Tests by their nature are perfectly fits for running in isolation. Theoretical limit of concurrent tests is millions, but for WebDriver tests problem is with avalilable browsers. Fortunately new hope is here – Selenoid, allows to scale your browsers farm to huge sizes. In this talk i want to show experiments about running NodeJS tests (ProtractorJS + MochaJS) into AWS Lambda and Selenoid as browser supplier.

Implicit vs Explicit waits is a holy-war topic ever. But let’s dig deeper than just words. Let’s go thought the code and find out why there are so many different thoughts and how to deal with all of them in near perfect harmony.

A test that was green yesterday can suddenly become red – for no apparent reason. Nobody changed anything. Just the moon is not in that phase. Just the matrix has you.

I will show my collection of real examples of flaky tests from my experience. These are really tricky investigations, that lasted many days, weeks and even years!

We’ll figure out how to write tests so that they are stable and independent of the karma of the developer.

My talk will be interesting to both testers and developers – anyone who loves to solve unsolvable puzzles.

Almost any application or software system manages data. It is hard to imagine test automation that is not affected by this fact. There are many differenct approaches how to prepare system under test, providing predefined test data: use application UI, invoke API methods, run business logic directly, access DB from test scenarios, etc. In this talk we will review most of existing approaches, starting from the easiest and the most popular ones and finishing with really tricky ways to manage your test data for large distributed systems. There is no ideal solution for every case or silver bullet, but I hope your toolset will become wider after visiting this talk.

With Agile adoption many things have changed in quality assurance and tester role. Ourdays the whole team is responsible for product quality. But not so many people understand how such high level approaches work in practice, how developer interacts with tester, what stages each task passes on the way from requirements specification to customer acceptance, who is doing what at each stage.

I have met only few teams, where developer and tester work closely together on a daily basis. Some projects try to same money on developer’s time, others try to have independent testing team without influence from developers side. Developers also don’t understad how tester could help them in practice. But this pair is able to significantly improve product quality and avoid many common issues.

In this talk we will cover motivation behind pair work of develoeper and tester, concrete practices and approaches at different stages, and advantages that both sides could achieve from such work style.

I would like to share basics about accessibility, my experience in this type of testing and also to talk about accessibility from automation perspective: can this type of testing be automated at all, what tools can be used, can accessibility testing be fully performed without people evaluation at all?

I will talk about integration testing nightmare (around 100 integrations with 3rd parties) and decisions that we did to get rid of integration testing. All talk is based on real project.

Nowadays, many organizations are challenging agile, and focus more on DevOps as their brand new ALM methodology. Devops can be regarded as a next generation methodology beyond agile, especially with it’s special focus on test automation.

If you want your organization to survive in today’s agile world, you need to learn and adapt to the changing landscape of Devops and utilize test automation to help speed delivery and high quality. When it comes to a successful DevOps transformation, test automation is one of the keys.

For some reason, some people are afraid of getting their hands dirty with Devops and test automation since they do believe that they don’t have the know-how, the tools, “enough” time, and, most commonly, the staff and budget. Of course, some approaches can be very difficult, take a lot of time, and be very expensive. But they don’t have to be. Within my session, I will discuss some of the test automation approaches we (as Keytorc) have successfully implemented in Devops environments and while doing so I will try to keep all the blocking factors as low as possible.

Once correctly implemented, these solutions may bring many benefits into your Devops environment. Some of the most critical ones include: a powerful foundation to cope with the software testing challenges and a systematic approach to shorten the time-to-market delivery by supporting the quality of your products in preliminary stages.

My session mainly focuses on the challenges of Devops and Test Automation and it will guide attendees to practically utilize different aspects of test automation based on the experiences of Devops and Agile.

In this talk, I am going to show approach to which helps to integrate test automation into delivery pipeline. DevOps practices become more popular and test automation engineers should follow them. I will show the way how we implemented flexible browser cluster, dockerized services on-demand and infrastructure as a code. Utilizing capabilities of Ansible, Selenoid, Jira test rail

Is it reasonable or not to control the quality of a test code? Why do Software Engineers in Test have to follow the same rules and conventions as developers do? Is there any tool which may help to partially automate and simplify the code review process? Should it be integrated into main development flow?

It’s time to change the way you look at Test Automation. In this talk you’ll see practical examples of the use of code quality tools, and understand how they can be applied in the CI flow.

Over the years, I’ve learned one key lesson from my experience in automating test cases. Regardless of how much effort I put into making my test suites the most well-designed, high-quality work of software engineering, the fact remains that all these test cases are useless if I don’t have a test infrastructure to adequately support running them. In my experience test environment availability is very limited and I need to be able to work with minimal resources.

For example, on a recent project I was responsible for, I had a suite of REST API tests, which needed to be run continuously in a collection of environments that mirrored a variety of customer environment configurations (OS, database server, application server, browser).

I could have gone the traditional route of having a permanent environment in place for each environment configuration but this would have been very draining on the limited resources available to me. Also, in my case, my test suite was made up of API tests, which meant they executed relatively quickly (compared to say UI tests). To have multiple environments dedicated to running a short test cycle a few times a day seemed to me like a massive waste of resources.

To solve this problem, I looked into using Docker containers. This way I could configure, create and remove environments easily, quickly so as best to suit our own environmental requirements.

So in this talk, I would like to take the audience through my journey of how I used Docker and CI tools to overcome the challenges I had around lack of test infrastructure resources.

In this talk I want to share our own experience testing on prod. Our users help us test and customize our algorithms and do not even know about it!

How it works?

Let’s discuss…

Powerful tools give a lot room for improvements. In order to make it valuable for you, I define your way of configuration. This talk will represent use cases of ReportPortal and the most beneficial configurations of usage. Based on real project examples, with collected metrics and efficiency improvement for them.

Internet changed our lives. We no longer go to book store since there is Amazon. We no longer watch TV since there are YouTube and Netflix. We tend to migrate into Internet and this is generates huge amount of traffic. This brings us new challenges but old solutions fail to keep pace with changes.

During past few years microservice oriented design became standard for distributed high load systems. It’s been proven by lots of companies including FANG (Facebook, Amazon, Netflix and Google) so at this stage efficiency of this approach is indisputable. Unfortunately there is no silver bullet and with all good parts we’ve received operation complexity, configuration management complexity, difficulties with debugging and tracing, instability of interfaces and performance drawback caused by expenses of IPC.

Goal of this talk is to get through performance analysis process of simple calculator application implemented as a set of microservices with following FOSS toolset: k6 and gatling for traffic generation, prometheus for monitoring, perf Linux profiler and FlameGraph project for flame graphs generation.

The talk is about a possibility of creation non-static code analyzer which will point a dev team to some unobvious bugs in the system based on machine learning technics.

We will become acquainted with Weka — the machine learning tool for exploratory development of the algorithm, also, we will take a closer look on basic algorithms for classification: Decision Trees, SVM and Naive Bayes classifier.

Real life examples will show you how the development life cycle can be improved with AI technics.

Basic functionalities of a peer 2 peer communication application are text, voice messaging, VoIP and video calls. In order to test its functionalities, we need more than two devices to communicate with each other. There are also network interruptions / changes or app notification installed on mobile device which can harm the quality of communication and application.

In order to automate regression test of those kind of application, we have extended Appium and Selenium Grid where we are able to cross communicate “n” number devices with each other. We also created a layer manipulating the mobile devices’ Bluetooth, wireless and other features.

Right now, we are able to automate P2P appliications test with more than 100 devices varying from iOS to Android.

Browser tests are known to be the flakiest ones. This is partly because browser infrastructure is complicated to maintain. But the second reason is – main stream browser automation tools such as Selenium server are far from being efficient.

As your company grows – your browser automation infrastructure should easily adapt to increased loads and new requirements. Unfortunately this is not so easy to get really scalable Selenium cluster using only the main stream tools. But there are some new open-source tools and approaches that can be used to deliver really scalable Selenium cluster.

This master-class covers an emerging browser automation tool – Selenoid, a truly efficient replacement of the standard Selenium server. I will explain how it works and why it is so fast. I will show in details features like video recording, sending logs and statistics to centralized storage and how to easily visualize this data.

Every automated testing system evolves over time. Let me tell you a story about how we started testing CloudForms. First with standard libraries and tools, then gradually developing and improving our own tools. Python played an important role, one which gave us the opportunities and freedom on how to approach the code. We subsequently did two refactors of the system. In the first, we focused on abstracting the UI interactions out of the test. Second, we carefully continued to iterate with feedback from our successes and failures. Instead of intertwined spaghetti code, we designed a couple of stand-alone libraries that jointly power our testing system with focus on declarativity. One library represents the abstraction of the UI, inspired by good ideas from Page Objects, but doing things better. The second one takes care of navigating around the UI in an intuitive way. And there are other ones as well. So let’s go through this story together and watch how we leveraged our Selenium testing with Python.