It is crucial once you have an automated test suite to run it often , get feedback as quick as possible and make the most of your tests to increase your team productivity and product quality. In our previous article we covered how to run selenium cucumber tests in chromedriver headless  but in order to have that done in a proper way we need to know how to run cucumber tests using docker in parallel .
Assuming we have docker setup already done in jenkins and your local machine if not here is how your docker file should look like we are going to use jenkins pipeline scripts to kick off our tests in parallel.
Go to Jenkins top page, select “New Job“, then choose “Build a free-style softwareproject“. This job type consists of the following elements: optional SCM, such as CVS or Subversion where your source code resides. optional triggers to control when Jenkins will perform builds.
Go to configure tab job :Â https://jenkins.yourjenkinshostname.com/view/{ViewName}/job/{JobName}/configure
Paste your pipeline script
stage 'Build'
node('QA-Slave') {
try {
currentBuild.description="Building Image"
git credentialsId: 'gitkey', url: 'kit project url'
docker.build "docker.yourhost.com/cucumber-autotest:${env.BUILD_NUMBER}"
}
catch (e) {
currentBuild.result="FAILED"
throw(e)
} finally {
notifier.notifyBuild(env.BUILD_NUMBER, currentBuild.description, currentBuild.result)
}
}
stage 'Test'
def runjob(String testexecutor,String profile){
currentBuild.description="Testing Image"
dockerContainerName="${testexecutor}-${env.BUILD_NUMBER}"
sh """
./bin/run_tests.sh ${dockerContainerName} ${"docker.yourhost.com/cucumber-autotest:${env.BUILD_NUMBER}"} -p ${profile} THREAD=${profile} -f pretty -f ReportPortal::Cucumber::Formatter -o 'output/test_report.txt';
"""
}
node('QA-Slave') {
timeout(time: 3, unit: 'HOURS'){
try{
parallel 'test1': {runjob('test1','thread1')},
'test2': {runjob('test2','thread2')},
'test3': {runjob('test3','thread3')},
'test4': {runjob('test4','thread4')},
'test5': {runjob('test5','thread5')},
'test6': {runjob('test6','thread6')},
'test7': {runjob('test7','thread7')},
'test8': {runjob('test8', 'thread8')},
'test9': {runjob('test9', 'thread9')},
'test10':{runjob('test10', 'thread10')}
}
catch (e) {
} finally {
step([$class: 'ArtifactArchiver', artifacts: 'output/**/*', excludes: null])
step([$class: 'JUnitResultArchiver', allowEmptyResults: true, keepLongStdio: true, testResults: 'output/*.xml'])
dockerCleanup();
if(notifier.getResults() >= 98){
currentBuild.result="SUCCESS"
sh "exit 0"
} else{
currentBuild.result="FAILED"
sh "exit 1"
throw new Exception("Pass rate lower than 98%")
}
deleteDir()
}
}
}
@NonCPS
def dockerCleanup() {
// Inspired by
// https://gitlab.zoopla.co.uk/devops/jenkins-libraries/blob/master/src/org/zpg/BasePipeline.groovy
// ... but 'mvn-smoke-tests' has made us cautious about over-cleansing
// So for now we'll do a simple `docker rmi`
// remove the *image* we created this build
// the container should auto-exit
sh """
docker rmi ${"docker.yourhost.com/cucumber-autotest:${env.BUILD_NUMBER}"};
"""
}
Lets try to understand some bits from this script:
You need it only if you use a notification system for your build test results , Im going to cover it in a separate article how you can send slack notifications using jenkins pipeline. I will do it separated as is a big topic.
Advertisements
In test stage you can see I used a function getresults()
if(notifier.getResults() >= 95){
currentBuild.result="SUCCESS"
sh "exit 0"
} else{
currentBuild.result="FAILED"
sh "exit 1"
throw new Exception("Pass rate lower than 95%")
}
Basically is our way to set a threshold in jenkins when to fail the build based in the test results. We automatically pass the build if fail rate is less than 2 %. I will cover how to do that in a further article.
Your run tests sh script
Inside runjob() function you’ve seem ./run_tests.sh , bellow you can see the code from it.
Using this jenkins pipeline script you can customise your cucumber tests to run on whatever environment you like , browser , brand ( if you have a big framework that incorporates multiple internal projects.)
Now there might be voices to say that we are defining the number of threads and we don’t leave  jenkins to split the objects dynamically . You can run jobs in parallel using the pipeline script provided by Jenkins but the problem with this will run all your tests sequentially first and only on the second run will be running in parallel.
I found it a bit weird because as you are going to increase the number of tests this job will run again and again your tests sequentially in order to figure out how to split the time equally.
// in this array we'll place the jobs that we wish to run
def branches = [:]
//running the job 4 times concurrently
//the dummy parameter is for preventing mutation of the parameter before the execution of the closure.
//we have to assign it outside the closure or it will run the job multiple times with the same parameter "4"
//and jenkins will unite them into a single run of the job
for (int i = 0; i < 4; i++) {
def index = i //if we tried to use i below, it would equal 4 in each job execution.
branches["branch${i}"] = {
//Parameters:
//param1 : an example string parameter for the triggered job.
//dummy: a parameter used to prevent triggering the job with the same parameters value. this parameter has to accept a different value
//each time the job is triggered.
build job: 'test_jobs', parameters: [[$class: 'StringParameterValue', name: 'param1', value:
'test_param'], [$class: 'StringParameterValue', name:'dummy', value: "${index}"]]
}
}
parallel branches
It’s up to you what solution you wanna choose , the top one works well for me.
If you have any queries please do not hesitate to add it in the comment bellow.
I have created a jenkins pipeline where in one of the stages I am running the scripts which runs the maven test cases then I copy that to the workspace the complete cucumber folder ,some of the cases are failing still build are marked as sucess
I have created a jenkins pipeline where in one of the stages I am running the scripts which runs the maven test cases then I copy that to the workspace the complete cucumber folder ,some of the cases are failing still build are marked as sucess
hi , it should not be the case
In the example I have a notified.getRresults()
My bad I didn’t published the code for the getResults method
here we go
@NonCPS
def getResults(){
AbstractTestResultAction testResultAction = currentBuild.rawBuild.getAction(AbstractTestResultAction.class)
def failCount = null
def failureDiffString = null
def totalCount = null
def passed = null
def passrate = null
if (testResultAction != null) {
failCount = testResultAction.failCount
totalCount = testResultAction.totalCount
passed = totalCount - failCount
passrate = (passed/totalCount*100).toInteger()
}
return passrate
}
also if you will have this code in a separate groovy file you need to create it
and load it in your pipeline
notifier = null
and inside your build after you checkout your code :
notifier = load 'jenkins/notify.groovy'
I will update the article with this informations