0

In an AWS EC2 serverm, OS: amzn2-ami-kernel-5.10 x86_64

Trying to diagnose an issue of a slow running cron script, identified a single line of command that took more than an hour to run, everyday.

mysql --login-path=something -DdDB --quick --skip-column-names --batch -e "set @var1=${VAR1}; set @var2='${VAR2}'; source sql.sql;" | tr '\t' ',' | tr -d '|' | sed 's/$/\r/' | sed 's/\\\\N/\\N/g' > "/mnt/somepath"

however when running the exact same command, with the same account that runs the script, it took no more than 10 seconds.

Is there any possible cause for this to happen? where a command takes much longer to run from a script triggered by crontab than from command line?

Improve this question
2
  • People tend to schedule batch jobs outside of business hours and that particular batch job may simply coincide with half a dozen other nightly batch jobs that put a large load on the server. Commented Oct 4, 2023 at 11:41
  • no thats not the case, it is scheduled at 4am, runs the command 8 times with different params, and expected to finish within an hour. Now the whole task starts at 4am, takes more than 12 hours to finish, with the 8 fire of the command scattered throughout the day from 4am until 9pm. I also overwatch all crontabs on the same server and those that connect to the same DB, there is no other slowness. I manually ran the command during office hour. Commented Oct 4, 2023 at 11:44

1 Answer 1

Reset to default
1

Turns out I did not recreate the exact command. What was shown in the question was the script, the command I ran tried to avoid var substitution

mysql --login-path=something -DdDB --quick --skip-column-names --batch -e "source sql_hardcoded_value.sql;" | tr '\t' ',' | tr -d '|' | sed 's/$/\r/' | sed 's/\\\\N/\\N/g' > "/mnt/somepath"

without the session variable being the parameter. that massively improved the performance.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.