Test is 6x slower in 3.3.2 with cy:open #26
Description
@jfaissolle commented on Fri Jun 28 2019
Current behavior:
With cypress open, running a single spec:
Running in 3.3.1 takes 21 seconds.
Running in 3.3.2 takes 117 seconds.
Most CPU is consumed by the hasBinary function.
Desired behavior:
Perfomance at least as good as 3.3.1 with cypress open.
Versions
Cypress 3.3.1 on Ubuntu 18.04.2, Chrome 75.0.3770.100
@flotwig commented on Fri Jun 28 2019
Hey @jfaissolle, could you share the spec code that is taking longer in 3.3.2? We made performance improvements in 3.3.2, but it looks like you're hitting a new edge case caused by #4407.
@jfaissolle commented on Mon Jul 01 2019
I have found that the slowdown happens with @cypress/code-coverage on. If I disable the plugin, tests perform as fast as in 3.3.1.
Here is an example of test.
const continueButton = '[data-ft="panel-footer"] button:contains("Continue")'; const backButton = '[data-ft="panel-footer"] button:contains("Back")'; const surveyCategory = '[data-ft="survey-category"]'; context('Stepper', function () { beforeEach(function () { cy.visit('iframe.html?id=survey--survey-stepper'); }); it('Looks good', function () { cy.wait(500); cy.matchImageSnapshot(); }); }); context('Actions', function () { beforeEach(function () { cy.visit('iframe.html?id=survey--survey-wizard'); cy.get('[data-ft="survey-card"]:first()').click(); }); it('Displays 3 steps', function () { cy.get('[data-ft="survey-stepper"] button').should('have.length', 3); }); it('Shows first category in survey', function () { cy.get(surveyCategory).should('contain', 'Actionnariat'); }); it('Clicking on second category should display second category', function () { cy.get('[data-ft="survey-stepper"] span:contains("Evolution organisation"):last()').click(); cy.get(surveyCategory).should('contain', 'Evolution organisation'); }); it('Clicking on Continue / Back should navigate in categories', function () { cy.get(continueButton).click(); cy.get(surveyCategory).should('contain', 'Evolution organisation'); cy.get(continueButton).click(); cy.get(surveyCategory).should('contain', 'Effectif'); cy.get(continueButton).should('not.exist'); cy.get(backButton).click(); cy.get(surveyCategory).should('contain', 'Evolution organisation'); }); it('Close with close button', function () { cy.get('[data-ft="panel-footer"] button:contains("Save")').click(); }); });@bahmutov commented on Mon Jul 01 2019
@jfaissolle yes, it can quite possibly be due to code-coverage plugin, because it combines the code coverage after each test with previous results - which includes loading a JSON file, merging it, saving updated JSON file. These are slow file operations and can be super slow for large applications where the coverage object is large. I wonder if hasBinary gets triggered when we use cy.task to send coverage info from the plugin to the Node process here https://github.com/cypress-io/code-coverage/blob/master/support.js and if we can pass a flag to NOT verify it (we know the object to be good for example)