Skip to content

PRJenkinsSetupFirewall

Jump to bottom
Josh Hursey edited this page Jan 18, 2017 · 11 revisions

Setting up a Jenkins Pull Request Builder from behind a firewall

If you have testing equipment behind a firewall then it can be difficult to use the Open MPI Community Jenkins server since it will not be able to reach your machines from the open internet.

Requirements

  • You will need to setup a Jenkins master internally to your site. We will call this (Internal)Jenkins.
  • Must be able to test internally without any external entity having access and visibility on the test environment or (Internal)Jenkins.
  • (Internal)Jenkins must be able to connect to the GitHub API.
  • (Internal)Jenkins must be able to report build result to the open internet so other developers can view details of build failures.
  • (Internal)Jenkins must be able to connect to the BuildNode, where you will run the tests.

Setup

Setting up the BuildNode

Setting up the environment on the BuildNode

# Setup a log file touch $PATH_TO_OMPI_TESTS/log/your-log.md # Push this file to the Gist site # Make sure to note the URL returned, as you will need the ID for later # --public makes it publicly available, remove that option if you want it private $PATH_TO_OMPI_TESTS/jenkins/bin/gist.pl --tokenfile $TOKENFILE -cmd create --public $PATH_TO_OMPI_TESTS/log/your-log.md # Setup the configuration for the relay system. Start with the template. cp $PATH_TO_OMPI_TESTS/jenkins/bin/config.inc.sample $PATH_TO_OMPI_TESTS/jenkins/bin/config.inc $EDITOR $PATH_TO_OMPI_TESTS/jenkins/bin/config.inc # Setup a crontab entry for the following script crontab -e # This script removes gists older than N days (Defined in config.inc) 0 8 * * * $PATH_TO_OMPI_TESTS/jenkins/bin/clean-history.sh $PATH_TO_OMPI_TESTS/jenkins/bin/config.inc

Setting up the (Internal)Jenkins Job

  • Define a default sha1 parameter and a default parameter that will be the default URL pushed to GitHub when the Jenkins job starts.
 -> "This build is parameterized" -> "String Parameter" -> Name: sha1 -> Default Value: master -> Description: -> "String Parameter" -> Name: GISTURL -> Default Value: https://gist.github.com -> Description: Initial URL for HitHub
  • (Suggested) Additional Behaviors
    • Wipe out repository & force clone
  • Configure the Pull Request Builder
 -> "GitHub Pull Request Builder" -> Define "Admin list" (add at least your bot account) -> "Advanced" -> Trigger phrase (replace 'ibm' with your org): .*bot:(ibm:)?retest.* -> Skip build phrase: .*((\[skip\W+ci\])|(\[ci\W+skip\])|(bot:notest)).* -> Crontab line (poll every 5 min): H/5 * * * * -> White list: open-mpi (your bot name) -> List of organizations. Their members will be whitelisted.: open-mpi -> (check) Build every pull request automatically without asking (Dangerous!). -> Whitelist Target Branches: master -> Poll SCM -> Schedule (should match earlier crontab line): H/5 * * * * -> "Trigger Setup" -> "Update commit status during build" # The "Commit Status Context" is what is displayed to GitHub next to the build status. # Usually seen as the "Details" link # So if you have multiple Jenkins jobs that run on a PR then distinguish them via # this string. Commit Status Context: YOUR-COMPANY-CI # This will be the URL posted next to the "Commit Status Context" on GitHub. # This environment variable is set in a .properties.txt file (in next step) Commit Status URL: ${GISTURL} Commit Status Build Triggered: Commit Status Build Started: -> "Build Status Messages" (optional if you want it to add a comment on failed builds) Build Result: Failure Message: "Test Failed! Some friendly string to put in the Comment field of the PR"
  • "Build" (in this order)
    • "Set build status to pending on GitHub commit"
    • "Execute shell"
#!/bin/bash -xe # This script does a few things: # - Post a placeholder file to Gist (we need the ID for later) # - Save the GISTID to a .properties.txt file (so we can update it later) # - Save the GISTURL to the .properties.txt file (so we can push it to GitHub later) # - Create a .env.sh with some information that we will need in the "Execute shell" phase below source $PATH_TO_OMPI_TESTS/jenkins/bin/pre-build.sh
  • "Inject environment variables"
    • Properties File Path: .properties.txt
  • "Execute shell"
#!/bin/bash -xe # Pick up the environment variables from the earlier "Execute shell" phase source $WORKSPACE/.env.sh #----------------------- # Redirect output to the file exec > >(tee -i output.txt) exec 2>&1 # Jenkins will fail this script at the point it has a non-zero exist status # so we need a file to see if we made it through the CI script successfully. touch $WORKSPACE/.in-progress.txt # Run your CI script here... # If we get here then everything is fine, so remove the file. rm $WORKSPACE/.in-progress.txt
  • "Post-build Actions"
    • "Execute a set of scripts"
      • "Build Steps" -> "Execute shell"
#!/bin/bash -x # This script does a few things # - Checks for the existance of the $WORKSPACE/.in-progress.txt to determine success/failure # - Updates the logfile $PATH_TO_OMPI_TESTS/log/your-log.md # - Pushes the log file to Gist # - Pushed the output file to Gist # - If the build was successful then the last N lines are sent (defined in config.inc) # - If the build was not successful then the entire output is sent source $PATH_TO_OMPI_TESTS/jenkins/bin/post-build.sh
  • "Set build status on GitHub commit"
    • Leave "Content" blank, Select "FAILURE" in "Result on failure"

Assorted notes (work in progress - not formatted correctly)

...

Clone this wiki locally