Feedback for Parallel Runner

[oleibman]

When I run, it says "Feedback highly appreciated". So here I go. I have enabled parallel runner (php-cs-fixer 3.59.3 php 8.3.8) on my Win11 system. The first time I analyze any new project (2,204 files), it gets to 98% almost instantly, and then stalls until it times out. Sometimes, rerunning it is sufficient to get it to work the second time. Sometimes it is necessary to turn off parallel runner to complete the analysis. Once the analysis has been completed by either method, I can turn parallel runner back on, and it will now work correctly for the rest of the duration of the project. I have not seen a similar problem in Linux.

[kubawerlos]

Hi @oleibman, can you share more details? Especially in a way described by Darek in the other issue: #8014 (comment)

From what you wrote it is impossible to take some action to fix it.

it gets to 98% almost instantly, and then stalls until it times out

this sounds like you have a huge file (files?). We have one really big file to test: https://github.com/PHP-CS-Fixer/PHP-CS-Fixer/blob/v3.59.3/tests/Fixtures/Integration/misc/meta_insert_64566_tokens.test-in.php - is that bigger or smaller than biggest of your 2204 files?

[oleibman]

Thanks, I knew the information I gave was inadequate, but wasn't sure what information you needed. I am using php-cs-fixer to analyze the files in PhpOffice/PhpSpreadsheet (src, samples, and test). There are currently 2,194 files with a total line count of 253,734. The largest file is 7,647 lines (Reader/Xls.php). In case it helps, there is only one other file > 4,000 lines; 3 more > 2,000; and 11 more > 1,000.

[Wirone]

@oleibman I was able to analyse PhpOffice/PhpSpreadsheet with parallel runner (MacBook Pro M1). It really stopped at 98%, then moved to 99% and was frozen like this for quite some time, but it finished correctly:

FYI: use --using-cache=no to always process everything (instant jump to 98% you mentioned most probably was an effect of a cache).

I believe progress formats from #6842 could help you with identifying files that take so long. I've rebased that branch some time ago so it's based on a parallel runner's version, you could use it for debugging. Anyway, it does not seem like a parallel runner issue, because the analysis freezes in the same way with --sequential mode, but it just takes even longer (194s comparing to 71s on 10 CPUs)

[oleibman]

Thank you. I was unable to figure out how to use the debugging options that you suggested. However, I was able to take a look at our largest module, which seems capable of at least some break-up. That's a little risky, but it can be justified for reasons besides this one. And doing it does seem to have a noticeable effect on overall time. I will take it slowly, but will continue to look into that option.

It appears that what might help in the short run while studying that option is a slight alteration to the config file:

setParallelConfig(PhpCsFixer\Runner\Parallel\ParallelConfigFactory::detect(null, 180)) 

That will raise the timeout limit. It seems a little obscure, but it obviously is find-able.

[NiklasBr]

I want to add some feedback, have been running with the parallel runner for a while using the below .php-cs-fixer.dist.php setting on a Symfony-based project, and it has been working fine as far as I can see.

PHP CS Fixer 3.70.0 Persian Successor by Fabien Potencier, Dariusz Ruminski and contributors.
PHP runtime: 8.3.16
Running analysis on 8 cores with 10 files per process.

<?php $finder = (new PhpCsFixer\Finder) ->in([ __DIR__ . '/bundles', __DIR__ . '/src', ]) ->exclude([ 'Migrations', ]) ; // do not enable self_accessor as it breaks pimcore models relying on get_called_class() return (new PhpCsFixer\Config) ->setRules([ '@PSR1' => true, '@PSR2' => true, 'array_syntax' => ['syntax' => 'short'], // keep aligned = and => operators as they are: do not force aligning, but do not remove it 'binary_operator_spaces' => ['operators' => ['=' => null, '=>' => null]], 'blank_line_before_statement' => ['statements' => ['return']], 'encoding' => true, 'function_typehint_space' => true, 'single_line_comment_style' => ['comment_types' => ['hash']], 'lowercase_cast' => true, 'magic_constant_casing' => true, 'method_argument_space' => ['on_multiline' => 'ignore'], 'class_attributes_separation' => ['elements' => ['method' => 'one']], 'modernize_strpos' => true, 'native_function_casing' => true, 'no_blank_lines_after_class_opening' => true, 'no_blank_lines_after_phpdoc' => true, 'no_empty_comment' => true, 'no_empty_phpdoc' => true, 'no_empty_statement' => true, 'no_extra_blank_lines' => true, 'no_leading_import_slash' => true, 'no_leading_namespace_whitespace' => true, 'no_short_bool_cast' => true, 'no_spaces_around_offset' => true, 'no_superfluous_phpdoc_tags' => ['allow_mixed' => true, 'remove_inheritdoc' => true], 'no_unneeded_control_parentheses' => true, 'no_unused_imports' => true, 'no_whitespace_before_comma_in_array' => true, 'no_whitespace_in_blank_line' => true, 'object_operator_without_whitespace' => true, 'ordered_imports' => true, 'phpdoc_indent' => true, 'phpdoc_no_useless_inheritdoc' => true, 'phpdoc_scalar' => true, 'phpdoc_separation' => true, 'phpdoc_single_line_var_spacing' => true, 'return_type_declaration' => true, 'short_scalar_cast' => true, 'single_blank_line_before_namespace' => true, 'single_quote' => true, 'space_after_semicolon' => true, 'standardize_not_equals' => true, 'ternary_operator_spaces' => true, 'whitespace_after_comma_in_array' => true, ]) ->setParallelConfig(PhpCsFixer\Runner\Parallel\ParallelConfigFactory::detect()) ->setFinder($finder);

[oleibman]

Everything has been working fine for quite some time now. I am closing this discussion.