BUG#35217728 Speedup mtr.pl by clean vardir

When using mtr.pl for development (running individual tests or parts of suites) the time spent setting up the environment far exceeds the time to run the actual test. Most timeconsuming are the steps to create the MySQL system database (i.e running mysqld --initialize) and copying the std_data directory into the var directory. Since the size of std_data grows over time as more and more stuff are added there, this step takes increasingly longer. Fix by improving the --fast mode to clean the var directory for reuse rather than removing it entirely and creating it again. This allows the data/ and std_data/ directories to be kept. Also make it possible to pass --no-fast to negate any previous --fast arguments one the command line. With this patch, the time to run a single simple test is decreased with 13 seconds on standard development machine. $ mtr alias <snip> Checking leftover processes Removing old var directory Creating var directory '/home/mblaudd/mysql/trunk/bld/mysql-test/var' - symlinking 'var' to '/dev/shm/var_auto_iXMr' Installing system database Using parallel: 1 ============================================================================== TEST NAME RESULT TIME (ms) ------------------------------------------------------------------------------ [ 50%] main.alias [ pass ] 758 [100%] shutdown_report [ pass ] ------------------------------------------------------------------------------ The servers were restarted 0 times The servers were reinitialized 0 times Spent 0.758 of 17 seconds executing testcases $ mtr alias --fast <snip> Checking leftover processes Cleaning var directory to save time Creating var directory '/home/mblaudd/mysql/trunk/bld/mysql-test/var' Reusing system database Using parallel: 1 ============================================================================== TEST NAME RESULT TIME (ms) COMMENT ------------------------------------------------------------------------------ [ 50%] main.alias [ pass ] 696 [100%] shutdown_report [ pass ] ------------------------------------------------------------------------------ The servers were restarted 0 times The servers were reinitialized 0 times Spent 0.696 of 4 seconds executing testcases Change-Id: Ibcf82874f61bbf1e1c8746da028045ccbc0a8a50

Author: blaudden

mysql-test/mysql-test-run.pl

@@ -1758,7 +1758,7 @@ sub command_line_setup {
  'default-myisam!' => \&collect_option,
  'disk-usage!' => \&report_option,
  'enable-disabled' => \&collect_option,
- 'fast' => \$opt_fast,
+ 'fast!' => \$opt_fast,
  'force-restart' => \$opt_force_restart,
  'help|h' => \$opt_usage,
  'keep-ndbfs' => \$opt_keep_ndbfs,
@@ -2084,6 +2084,7 @@ sub command_line_setup {
  # fast option
  if ($opt_fast) {
  # Kill processes instead of nice shutdown
+ mtr_report("Using kill instead of nice shutdown (--fast)");
  $opt_shutdown_timeout = 0;
  }
 
@@ -3407,6 +3408,34 @@ ()
 
 # Remove var and any directories in var/ created by previous tests
 sub remove_stale_vardir () {
+
+ if ($opt_fast) {
+ mtr_report(" * NOTE! Using --fast for quicker development testing.");
+ mtr_report(" * This may cause spurious test failures if contents");
+ mtr_report(" * of std_data/ or data/ does not match the tests or");
+ mtr_report(" * compiled binaries. Fix by removing vardir (or run");
+ mtr_report(" * once without --fast).");
+
+ # Just clean the vardir, this allows reusing data/ and std_data/
+ # which otherwise takes time to create or copy.
+ mtr_report("Cleaning var directory to save time (--fast)");
+ foreach my $name (glob("$opt_vardir/*")) {
+ if (!-l $name && -d _) {
+ if ($name =~ /\/data$/ ||
+ $name =~ /\/std_data$/) {
+ # Preserve directory
+ next;
+ }
+ # Remove directory
+ rmtree($name);
+ next;
+ }
+ # Remove file
+ unlink($name);
+ }
+ return;
+ }
+
  mtr_report("Removing old var directory");
 
  mtr_error("No, don't remove the vardir when running with --extern")
@@ -3517,7 +3546,8 @@ ()
 
  # Copy all files from std_data into var/std_data
  # and make them world readable
- copytree("$glob_mysql_test_dir/std_data", "$opt_vardir/std_data", "0022");
+ copytree("$glob_mysql_test_dir/std_data", "$opt_vardir/std_data", "0022")
+ unless -d "$opt_vardir/std_data";
 
  # Remove old log files
  foreach my $name (glob("r/*.progress r/*.log r/*.warnings")) {
@@ -4155,8 +4185,6 @@ sub mysql_install_db {
  my $install_chsdir = $mysqld->value('character-sets-dir');
  my $install_datadir = $datadir || $mysqld->value('datadir');
 
- mtr_report("Installing system database");
-
  my $args;
  mtr_init_args(\$args);
  mtr_add_arg($args, "--no-defaults");
@@ -4213,6 +4241,14 @@ sub mysql_install_db {
  # Export MYSQLD_INSTALL_CMD variable containing <path>/mysqld <args>
  $ENV{'MYSQLD_INSTALL_CMD'} = "$exe_mysqld_bootstrap " . join(" ", @$args);
 
+ if ($opt_fast && -d $datadir) {
+ mtr_report("Reusing system database (--fast)");
+
+ return;
+ }
+
+ mtr_report("Installing system database");
+
  # Create the bootstrap.sql file
  my $bootstrap_sql_file = "$opt_vardir/tmp/bootstrap.sql";
 
@@ -7975,8 +8011,9 @@ ($)
  tests. This is needed after switching default storage
  engine to InnoDB.
  disk-usage Show disk usage of vardir after each test.
- fast Run as fast as possible, dont't wait for servers
- to shutdown etc.
+ fast Run mtr.pl as fast as possible when using it for
+ development. This involves reusing the vardir (just
+ clean it) and don't wait for servers to shutdown.
  force-restart Always restart servers between tests.
  gcov Collect coverage information after the test.
  The result is a gcov file per source and header file.