<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>jenkins Archives - Tech Chronicles</title>
	<atom:link href="http://kostacipo.stream/tag/jenkins/feed/" rel="self" type="application/rss+xml" />
	<link>http://kostacipo.stream/tag/jenkins/</link>
	<description>Ramblings of a Tech Dude</description>
	<lastBuildDate>Mon, 13 Apr 2020 12:05:34 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=6.8.2</generator>

<image>
	<url>https://kostacipo.stream/wp-content/uploads/2019/12/cropped-profile-32x32.jpg</url>
	<title>jenkins Archives - Tech Chronicles</title>
	<link>http://kostacipo.stream/tag/jenkins/</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>SonarQube Integration with Jenkins Using Pipelines</title>
		<link>http://kostacipo.stream/sonarqube-integration-with-jenkins-using-pipelines/</link>
					<comments>http://kostacipo.stream/sonarqube-integration-with-jenkins-using-pipelines/#respond</comments>
		
		<dc:creator><![CDATA[Majordomo]]></dc:creator>
		<pubDate>Mon, 13 Apr 2020 12:05:34 +0000</pubDate>
				<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Tools]]></category>
		<category><![CDATA[jenkins]]></category>
		<category><![CDATA[sonarqube]]></category>
		<guid isPermaLink="false">http://www.kostacipo.stream/?p=1779</guid>

					<description><![CDATA[<p>SonarQube Integration is an open source static code analysis tool that is gaining tremendous popularity among software developers. It enables software professionals to measure code quality, identify non-compliant code, and fix code quality issues. The SonarQube community is quite active and provides continuous upgrades, new plug-ins, and customization information on a regular basis. Further, it [&#8230;]</p>
<p>The post <a href="http://kostacipo.stream/sonarqube-integration-with-jenkins-using-pipelines/">SonarQube Integration with Jenkins Using Pipelines</a> appeared first on <a href="http://kostacipo.stream">Tech Chronicles</a>.</p>
]]></description>
										<content:encoded><![CDATA[<p style="text-align: justify;">SonarQube Integration is an open source static code analysis tool that is gaining tremendous popularity among software developers. It enables software professionals to measure code quality, identify non-compliant code, and fix code quality issues. The SonarQube community is quite active and provides continuous upgrades, new plug-ins, and customization information on a regular basis. Further, it is a healthy practice to periodically run SonarQube on the source code to fix code quality violations and reduce the technical debt.</p>
<p style="text-align: justify;">SonarQube enables developers to track code quality, which helps them to ascertain if a project is ready to be deployed in production. Further, it allows developers to continuously inspect the code, perform automatic reviews and run analysis to find code quality issues.</p>
<p style="text-align: justify;">Furthermore, SonarQube provides a lot of other features, including the ability to record metrics, evolution graphs etc. It has inherent options to perform automated analysis and <a href="https://en.wikipedia.org/wiki/Continuous_integration" target="_blank" rel="noopener noreferrer">continuous integration</a> utilizing tools such as <a href="https://jenkins-ci.org/" target="_blank" rel="noopener noreferrer">Jenkins</a>, <a href="http://hudson-ci.org/" target="_blank" rel="noopener noreferrer">Hudson</a>, etc. In this blog, we will explore the process of creating pipeline scripts for SonarQube integration. Here are the steps.</p>
<ul style="text-align: justify;">
<li>Log in to your configuration domain (e.g. https://jenkins.domain.com).</li>
<li>Go to the specific profile in Jenkins (Here, in this case, it is the root project).</li>
<li>To configure a Sonar job, select <em>‘New Item’</em> available on the left side panel in Jenkins.</li>
</ul>
<p style="text-align: justify;"><img decoding="async" class="aligncenter size-full wp-image-3490 lazyloaded" src="https://i2.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/New-Item.png?resize=154%2C168&amp;ssl=1" alt="New Item" data-recalc-dims="1" data-lazy-src="https://i2.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/New-Item.png?resize=154%2C168&amp;ssl=1" data-was-processed="true" width="154" height="168"></p>
<ul style="text-align: justify;">
<li>In the subsequent screen provide a job name. Click on <em>‘Pipeline’</em> option, if you intend to run a Pipeline, else select the <em>‘Maven’</em> option.</li>
</ul>
<p style="text-align: justify;"><img fetchpriority="high" decoding="async" class="aligncenter size-full wp-image-3492 lazyloaded" src="https://i0.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/Pipeline-Maven.png?resize=380%2C370&amp;ssl=1" sizes="(max-width: 380px) 100vw, 380px" srcset="https://i0.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Pipeline-Maven.png?w=380&amp;ssl=1 380w, https://i0.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Pipeline-Maven.png?resize=300%2C292&amp;ssl=1 300w" alt="Pipeline Maven" data-lazy-srcset="https://i0.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Pipeline-Maven.png?w=380&amp;ssl=1 380w, https://i0.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Pipeline-Maven.png?resize=300%2C292&amp;ssl=1 300w" data-lazy-sizes="(max-width: 380px) 100vw, 380px" data-recalc-dims="1" data-lazy-src="https://i0.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/Pipeline-Maven.png?resize=380%2C370&amp;ssl=1" data-was-processed="true" width="380" height="370"></p>
<ul style="text-align: justify;">
<li>Click on <em>‘OK’</em> button to provide configuration details. Now, let us take a look at the various links available on the left side of the screen.
<ul>
<li>Changes – The <em>‘Changes’</em> option enables developers to change the name of a job.</li>
<li>Build Now – This feature allows developers to run a job in Jenkins. It starts to read the code from the&nbsp;repository and builds the code.</li>
<li>Configure – This option enables developers to read the code from the Git/SVN repository.</li>
</ul>
</li>
</ul>
<p style="text-align: justify;"><img decoding="async" class="aligncenter size-full wp-image-3493 lazyloaded" src="https://i1.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/Pipeline-inc-web.png?resize=622%2C283&amp;ssl=1" sizes="(max-width: 622px) 100vw, 622px" srcset="https://i1.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Pipeline-inc-web.png?w=622&amp;ssl=1 622w, https://i1.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Pipeline-inc-web.png?resize=300%2C136&amp;ssl=1 300w" alt="Pipeline inc-web" data-lazy-srcset="https://i1.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Pipeline-inc-web.png?w=622&amp;ssl=1 622w, https://i1.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Pipeline-inc-web.png?resize=300%2C136&amp;ssl=1 300w" data-lazy-sizes="(max-width: 622px) 100vw, 622px" data-recalc-dims="1" data-lazy-src="https://i1.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/Pipeline-inc-web.png?resize=622%2C283&amp;ssl=1" data-was-processed="true" width="622" height="283"></p>
<h3 style="text-align: justify;">SonarQube Integration with Jenkins</h3>
<p style="text-align: justify;">Here is the complete process of&nbsp;SonarQube integration with Jenkins.</p>
<ul style="text-align: justify;">
<li>Click on <em>‘Configure’</em> option, which will redirect developers to the following screen, enabling them to read the code from the Git/SVN repository.</li>
<li>In the General tab, developers can provide a Pipeline name and log build details, such as how many days the logs should be kept etc. In the <em>‘Days to keep builds’</em> field, enter the&nbsp;number of days.</li>
</ul>
<p style="text-align: justify;"><img loading="lazy" decoding="async" class="aligncenter size-full wp-image-3494 lazyloaded" src="https://i0.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/General.png?resize=638%2C388&amp;ssl=1" sizes="auto, (max-width: 638px) 100vw, 638px" srcset="https://i0.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/General.png?w=638&amp;ssl=1 638w, https://i0.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/General.png?resize=300%2C182&amp;ssl=1 300w" alt="General" data-lazy-srcset="https://i0.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/General.png?w=638&amp;ssl=1 638w, https://i0.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/General.png?resize=300%2C182&amp;ssl=1 300w" data-lazy-sizes="(max-width: 638px) 100vw, 638px" data-recalc-dims="1" data-lazy-src="https://i0.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/General.png?resize=638%2C388&amp;ssl=1" data-was-processed="true" width="638" height="388"></p>
<ul style="text-align: justify;">
<li>If using a Git repository, select Git project, else proceed to the next tab.</li>
<li>The next tab is ‘<em>Office 365 Connector</em>’, this screen is used to run a build based on parameters.</li>
<li>The option <em>‘This project is parameterized’</em> is one of the parameters that developers should have a brief idea. It is used to build a job based on branches. In situations, where developers want to run a job based on development/master branches, they can define it using this option else, the job will run as default.</li>
</ul>
<p style="text-align: justify;"><img loading="lazy" decoding="async" class="aligncenter size-full wp-image-3521 lazyloaded" src="https://i0.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/Office-365-Connector-1.png?resize=637%2C297&amp;ssl=1" sizes="auto, (max-width: 637px) 100vw, 637px" srcset="https://i1.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Office-365-Connector-1.png?w=637&amp;ssl=1 637w, https://i1.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Office-365-Connector-1.png?resize=300%2C140&amp;ssl=1 300w" alt="Office 365 Connector" data-lazy-srcset="https://i1.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Office-365-Connector-1.png?w=637&amp;ssl=1 637w, https://i1.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Office-365-Connector-1.png?resize=300%2C140&amp;ssl=1 300w" data-lazy-sizes="(max-width: 637px) 100vw, 637px" data-recalc-dims="1" data-lazy-src="https://i0.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/Office-365-Connector-1.png?resize=637%2C297&amp;ssl=1" data-was-processed="true" width="637" height="297"></p>
<ul style="text-align: justify;">
<li><strong>Build Trigger Section</strong> – This option can be utilized, if developers are keen to run the jobs based on specific time intervals e.g. once a day or twice a day. Additionally, jobs can be scheduled to run automatically by using the <em>‘Build periodically’</em> option.</li>
<li><strong>Pipeline Section</strong> – The pipeline section is the core feature of a job; it reads data/code from a specific repository in GIT/SVN. However, developers need to specify the script file, which is available in the GIT/SVN application.</li>
</ul>
<p style="text-align: justify;"><img loading="lazy" decoding="async" class="aligncenter size-full wp-image-3522 lazyloaded" src="https://i0.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/Pipeline-1.png?resize=625%2C346&amp;ssl=1" sizes="auto, (max-width: 625px) 100vw, 625px" srcset="https://i1.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Pipeline-1.png?w=625&amp;ssl=1 625w, https://i1.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Pipeline-1.png?resize=300%2C166&amp;ssl=1 300w" alt="Pipeline" data-lazy-srcset="https://i1.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Pipeline-1.png?w=625&amp;ssl=1 625w, https://i1.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Pipeline-1.png?resize=300%2C166&amp;ssl=1 300w" data-lazy-sizes="(max-width: 625px) 100vw, 625px" data-recalc-dims="1" data-lazy-src="https://i0.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/Pipeline-1.png?resize=625%2C346&amp;ssl=1" data-was-processed="true" width="625" height="346"></p>
<ul style="text-align: justify;">
<li>Here is the pipeline script that needs to be added to the Jenkins file.</li>
</ul>
<div style="text-align: justify;">
<div id="highlighter_603459" class="syntaxhighlighter  java">
<div class="toolbar"><a class="toolbar_item command_help help" href="https://www.evoketechnologies.com/blog/sonarqube-integration-jenkins-pipelines/#">?</a></div>
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td class="gutter">
<div class="line number1 index0 alt2">1</div>
<div class="line number2 index1 alt1">2</div>
<div class="line number3 index2 alt2">3</div>
<div class="line number4 index3 alt1">4</div>
<div class="line number5 index4 alt2">5</div>
<div class="line number6 index5 alt1">6</div>
<div class="line number7 index6 alt2">7</div>
<div class="line number8 index7 alt1">8</div>
</td>
<td class="code">
<div class="container">
<div class="line number1 index0 alt2"><code class="java keyword">try</code> <code class="java plain">{</code></div>
<div class="line number2 index1 alt1"><code class="java plain">stage(</code><code class="java string">"Building SONAR ..."</code><code class="java plain">) {</code></div>
<div class="line number3 index2 alt2"><code class="java plain">sh </code><code class="java string">'./gradlew clean sonarqube'</code></div>
<div class="line number4 index3 alt1"><code class="java plain">}</code></div>
<div class="line number5 index4 alt2"><code class="java plain">} </code><code class="java keyword">catch</code> <code class="java plain">(e) {emailext attachLog: </code><code class="java keyword">true</code><code class="java plain">, body: </code><code class="java string">'See attached log'</code><code class="java plain">, subject: </code><code class="java string">'BUSINESS Build Failure'</code><code class="java plain">, to: </code><code class="java string">'abc@gmail.com'</code></div>
<div class="line number6 index5 alt1"><code class="java plain">step([$</code><code class="java keyword">class</code><code class="java plain">: </code><code class="java string">'WsCleanup'</code><code class="java plain">])</code></div>
<div class="line number7 index6 alt2"><code class="java keyword">return</code></div>
<div class="line number8 index7 alt1"><code class="java plain">}</code></div>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<ul style="text-align: justify;">
<li>Add the following plugin details in the build.gradle/pom.xml file (if it is maven):</li>
</ul>
<p style="text-align: justify;">a) Sonar Plugin</p>
<div style="text-align: justify;">
<div id="highlighter_651812" class="syntaxhighlighter  java">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td class="gutter">
<div class="line number1 index0 alt2">1</div>
<div class="line number2 index1 alt1">2</div>
<div class="line number3 index2 alt2">3</div>
</td>
<td class="code">
<div class="container">
<div class="line number1 index0 alt2"><code class="java plain">dependencies {</code></div>
<div class="line number2 index1 alt1"><code class="java plain">classpath </code><code class="java string">"org.sonarsource.scanner.gradle: sonarqube-gradle-plugin:2.5"</code></div>
<div class="line number3 index2 alt2"><code class="java plain">}</code></div>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<p style="text-align: justify;">b) Apply plugin: ‘org.sonarqube’</p>
<p style="text-align: justify;">c) Add SonarQube Server Details:</p>
<div style="text-align: justify;">
<div id="highlighter_478216" class="syntaxhighlighter  java">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td class="gutter">
<div class="line number1 index0 alt2">1</div>
<div class="line number2 index1 alt1">2</div>
<div class="line number3 index2 alt2">3</div>
<div class="line number4 index3 alt1">4</div>
<div class="line number5 index4 alt2">5</div>
<div class="line number6 index5 alt1">6</div>
<div class="line number7 index6 alt2">7</div>
</td>
<td class="code">
<div class="container">
<div class="line number1 index0 alt2"><code class="java plain">sonarqube {</code></div>
<div class="line number2 index1 alt1"><code class="java plain">properties {</code></div>
<div class="line number3 index2 alt2"><code class="java plain">property </code><code class="java string">"sonar.host.url"</code><code class="java plain">, http:</code><code class="java comments">//sonar.xxxxx.com //&nbsp; url is your sonar server</code></div>
<div class="line number4 index3 alt1"><code class="java plain">property </code><code class="java string">"sonar.projectName"</code><code class="java plain">, </code><code class="java string">"project&nbsp; display name"</code>&nbsp;&nbsp; <code class="java comments">//&nbsp; this name will appear in dashboard</code></div>
<div class="line number5 index4 alt2"><code class="java plain">property </code><code class="java string">"sonar.projectKey"</code><code class="java plain">, </code><code class="java string">"projectKey"</code> <code class="java comments">// It sould be a keybased on this report is created</code></div>
<div class="line number6 index5 alt1"><code class="java plain">property </code><code class="java string">"sonar.groovy.jacoco.reportPath"</code><code class="java plain">, </code><code class="java string">"${project.buildDir}/jacoco/test.exec"</code>&nbsp;&nbsp;&nbsp; <code class="java plain">}</code></div>
<div class="line number7 index6 alt2"><code class="java plain">}</code></div>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<ul style="text-align: justify;">
<li>Once the configuration is complete, developers can build job manually or automatically. This can be activated using the option ‘<em>Build Now</em>’ available on left side panel of the screen.</li>
</ul>
<p style="text-align: justify;"><img loading="lazy" decoding="async" class="aligncenter size-full wp-image-3498 lazyloaded" src="https://i2.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/Build-Job.png?resize=204%2C229&amp;ssl=1" alt="Build Job" data-recalc-dims="1" data-lazy-src="https://i2.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/Build-Job.png?resize=204%2C229&amp;ssl=1" data-was-processed="true" width="204" height="229"></p>
<ul style="text-align: justify;">
<li>After the build process is complete in Jenkins, it reads the code and compares each line, if it observes any violations of rules, it sends a report to the sonar server. Here is a sample snapshot of the SonarQube report.</li>
</ul>
<p style="text-align: justify;"><img loading="lazy" decoding="async" class="aligncenter size-full wp-image-3499 lazyloaded" src="https://i0.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/Java-Maven-Project.png?resize=641%2C448&amp;ssl=1" sizes="auto, (max-width: 641px) 100vw, 641px" srcset="https://i0.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Java-Maven-Project.png?w=641&amp;ssl=1 641w, https://i0.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Java-Maven-Project.png?resize=300%2C210&amp;ssl=1 300w" alt="Java Maven Project" data-lazy-srcset="https://i0.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Java-Maven-Project.png?w=641&amp;ssl=1 641w, https://i0.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Java-Maven-Project.png?resize=300%2C210&amp;ssl=1 300w" data-lazy-sizes="(max-width: 641px) 100vw, 641px" data-recalc-dims="1" data-lazy-src="https://i0.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/Java-Maven-Project.png?resize=641%2C448&amp;ssl=1" data-was-processed="true" width="641" height="448"></p>
<p style="text-align: justify;">Developers can view a list of issues on the SonarQube dashboard. If they are interested to find out what went wrong in their code base, all they have to do it simply click on specific links (numbers above). This action will redirect developers to specific code, where they can fix the issues.</p>
<p style="text-align: justify;">Here is an example, below is a test class, where we have created a sample Java class.</p>
<p style="text-align: justify;"><img loading="lazy" decoding="async" class="aligncenter size-full wp-image-3500 lazyloaded" src="https://i1.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/Sample-Java-Class.png?resize=633%2C452&amp;ssl=1" sizes="auto, (max-width: 633px) 100vw, 633px" srcset="https://i1.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Sample-Java-Class.png?w=633&amp;ssl=1 633w, https://i1.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Sample-Java-Class.png?resize=300%2C214&amp;ssl=1 300w" alt="Sample Java Class" data-lazy-srcset="https://i1.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Sample-Java-Class.png?w=633&amp;ssl=1 633w, https://i1.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Sample-Java-Class.png?resize=300%2C214&amp;ssl=1 300w" data-lazy-sizes="(max-width: 633px) 100vw, 633px" data-recalc-dims="1" data-lazy-src="https://i1.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/Sample-Java-Class.png?resize=633%2C452&amp;ssl=1" data-was-processed="true" width="633" height="452"></p>
<p style="text-align: justify;">If you observe, Code Smells count is 3, clicking on number 3 will redirect developers to the following screen on a Sonar server. Further, it will show/suggest the vulnerability based on the rule.</p>
<p style="text-align: justify;"><img loading="lazy" decoding="async" class="aligncenter size-full wp-image-3501 lazyloaded" src="https://i1.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/Code-smells-count.png?resize=639%2C199&amp;ssl=1" sizes="auto, (max-width: 639px) 100vw, 639px" srcset="https://i2.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Code-smells-count.png?w=639&amp;ssl=1 639w, https://i2.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Code-smells-count.png?resize=300%2C93&amp;ssl=1 300w" alt="Code smells count" data-lazy-srcset="https://i2.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Code-smells-count.png?w=639&amp;ssl=1 639w, https://i2.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/Code-smells-count.png?resize=300%2C93&amp;ssl=1 300w" data-lazy-sizes="(max-width: 639px) 100vw, 639px" data-recalc-dims="1" data-lazy-src="https://i1.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/Code-smells-count.png?resize=639%2C199&amp;ssl=1" data-was-processed="true" width="639" height="199"></p>
<p style="text-align: justify;">In our code ‘testService.java’, we have used a sample system.out .print ln( ) method. In Sonar server, a rule is defined that mentions use logger instead of system.out.</p>
<p style="text-align: justify;">The below method main() is kept empty in ‘my testservice.java class’, as can be observed, SonarQube is recommending to comment on this method since this method is empty. Similarly, it shows other issues in the code.&nbsp;</p>
<p style="text-align: justify;"><img loading="lazy" decoding="async" class="aligncenter size-full wp-image-3502 lazyloaded" src="https://i1.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/testservice.png?resize=636%2C126&amp;ssl=1" sizes="auto, (max-width: 636px) 100vw, 636px" srcset="https://i0.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/testservice.png?w=636&amp;ssl=1 636w, https://i0.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/testservice.png?resize=300%2C59&amp;ssl=1 300w" alt="testservice" data-lazy-srcset="https://i0.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/testservice.png?w=636&amp;ssl=1 636w, https://i0.wp.com/www.evoketechnologies.com/wp-content/uploads/2017/12/testservice.png?resize=300%2C59&amp;ssl=1 300w" data-lazy-sizes="(max-width: 636px) 100vw, 636px" data-recalc-dims="1" data-lazy-src="https://i1.wp.com/d331tpl5vusgqa.cloudfront.net/wp-content/uploads/2017/12/testservice.png?resize=636%2C126&amp;ssl=1" data-was-processed="true" width="636" height="126"></p>
<p style="text-align: justify;">Hope this post serves the purpose of providing insights on SonarQube integration, if you have any specific questions or comments, please feel free to post your comments.</p>
<p>The post <a href="http://kostacipo.stream/sonarqube-integration-with-jenkins-using-pipelines/">SonarQube Integration with Jenkins Using Pipelines</a> appeared first on <a href="http://kostacipo.stream">Tech Chronicles</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>http://kostacipo.stream/sonarqube-integration-with-jenkins-using-pipelines/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>Automating PostgreSQL Daily Tasks Using Jenkins</title>
		<link>http://kostacipo.stream/automating-postgresql-daily-tasks-using-jenkins/</link>
					<comments>http://kostacipo.stream/automating-postgresql-daily-tasks-using-jenkins/#respond</comments>
		
		<dc:creator><![CDATA[Majordomo]]></dc:creator>
		<pubDate>Fri, 31 Jan 2020 10:55:29 +0000</pubDate>
				<category><![CDATA[DevOps]]></category>
		<category><![CDATA[databases]]></category>
		<category><![CDATA[jenkins]]></category>
		<category><![CDATA[postgresql]]></category>
		<guid isPermaLink="false">http://www.kostacipo.stream/?p=1729</guid>

					<description><![CDATA[<p>&#160; About Jenkins Jenkins&#160;is an open source software for automation. It is developed in java and is one of the most popular tools for Continuous Integration (CI) and Continuous Delivery (CD). In 2010, after the acquisition of Sun Microsystems by Oracle, &#8220;Hudson&#8221; software was in a dispute with its open source community. This dispute became [&#8230;]</p>
<p>The post <a href="http://kostacipo.stream/automating-postgresql-daily-tasks-using-jenkins/">Automating PostgreSQL Daily Tasks Using Jenkins</a> appeared first on <a href="http://kostacipo.stream">Tech Chronicles</a>.</p>
]]></description>
										<content:encoded><![CDATA[<p>&nbsp;</p>
<h2>About Jenkins</h2>
<p><a href="https://en.wikipedia.org/wiki/Jenkins_(software)" target="_blank" rel="nofollow noopener noreferrer">Jenkins</a>&nbsp;is an open source software for automation. It is developed in java and is one of the most popular tools for Continuous Integration (CI) and Continuous Delivery (CD).</p>
<p>In 2010, after the acquisition of Sun Microsystems by Oracle, &#8220;<a href="https://en.wikipedia.org/wiki/Hudson_(software)" target="_blank" rel="nofollow noopener noreferrer">Hudson</a>&#8221; software was in a dispute with its open source community. This dispute became the basis for the launch of the Jenkins project.</p>
<p>Nowadays, &#8220;Hudson” (Eclipse public license) and “Jenkins” (MIT license) are two active and independent projects with a very similar purpose.</p>
<p>Jenkins has thousands of plugins you can use in order to speed up the development phase through automation for the entire development life-cycle; build, document, test, package, stage and deployment.</p>
<h2>What Does Jenkins Do?</h2>
<p>Although the main use of Jenkins could be Continuous Integration (CI) and Continuous Delivery (CD), this open source has a set of functionalities and it can be used without any commitment or dependence from CI or CD, thus Jenkins presents some interesting functionalities to explore:</p>
<ul>
<li>Scheduling period jobs (instead of using the traditional&nbsp;<em>crontab</em>)</li>
<li>Monitoring jobs, its logs and activities by a clean view (as they have an option for grouping)</li>
<li>Maintenance of jobs could be done easily; assuming Jenkins has a set of options for it</li>
<li>Setup and scheduling software installation (by using Puppet) in the same host or in another one.</li>
<li>Publishing reports and sending email notifications</li>
</ul>
<h2>Running PostgreSQL Tasks in Jenkins</h2>
<p>There are three common tasks a&nbsp;PostgreSQL&nbsp;developer or database administrator has to do on a daily basis:</p>
<ol>
<li>Scheduling and execution of PostgreSQL scripts</li>
<li>Executing a PostgreSQL process composed of three or more scripts</li>
<li>Continuous Integration (CI) for PL/pgSQL developments</li>
</ol>
<p>For the execution of these examples, it is assumed that Jenkins and PostgreSQL (at least the version 9.5) servers are installed and working properly.</p>
<h2>Scheduling and Execution of a PostgreSQL Script</h2>
<p>In most cases the implementation of daily (or periodically) PostgreSQL scripts for the execution of a usual task such as&#8230;</p>
<ul>
<li>Generation of backups</li>
<li>Test the restore of a backup</li>
<li>Execution of a query for reporting purposes</li>
<li>Clean up and archiving log files</li>
<li>Calling a PL/pgSQL procedure to purge tables</li>
</ul>
<p>t’s defined on&nbsp;<em>crontab</em>:</p>
<div>
<div id="highlighter_924287" class="syntaxhighlighter  bash">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td class="gutter">
<div class="line number1 index0 alt2">1</div>
<div class="line number2 index1 alt1">2</div>
<div class="line number3 index2 alt2">3</div>
<div class="line number4 index3 alt1">4</div>
<div class="line number5 index4 alt2">5</div>
<div class="line number6 index5 alt1">6</div>
<div class="line number7 index6 alt2">7</div>
<div class="line number8 index7 alt1">8</div>
<div class="line number9 index8 alt2">9</div>
</td>
<td class="code">
<div class="container">
<div class="line number1 index0 alt2"><code class="bash plain">0 5,17 * * * </code><code class="bash plain">/filesystem/scripts/archive_logs</code><code class="bash plain">.sh</code></div>
<div class="line number2 index1 alt1"><code class="bash plain">0 2 * * * </code><code class="bash plain">/db/scripts/db_backup</code><code class="bash plain">.sh</code></div>
<div class="line number3 index2 alt2"><code class="bash plain">0 6 * * * </code><code class="bash plain">/db/data/scripts/backup_client_tables</code><code class="bash plain">.sh</code></div>
<div class="line number4 index3 alt1"><code class="bash plain">0 4 * * * </code><code class="bash plain">/db/scripts/Test_db_restore</code><code class="bash plain">.sh</code></div>
<div class="line number5 index4 alt2"><code class="bash plain">*</code><code class="bash plain">/10</code> <code class="bash plain">* * * * </code><code class="bash plain">/db/scripts/monitor</code><code class="bash plain">.sh</code></div>
<div class="line number6 index5 alt1"><code class="bash plain">0 4 * * * </code><code class="bash plain">/db/data/scripts/queries</code><code class="bash plain">.sh</code></div>
<div class="line number7 index6 alt2"><code class="bash plain">0 4 * * * </code><code class="bash plain">/db/scripts/data_extraction</code><code class="bash plain">.sh</code></div>
<div class="line number8 index7 alt1"><code class="bash plain">0 5 * * * </code><code class="bash plain">/db/scripts/data_import</code><code class="bash plain">.sh</code></div>
<div class="line number9 index8 alt2"><code class="bash plain">0 *</code><code class="bash plain">/4</code> <code class="bash plain">* * * </code><code class="bash plain">/db/data/scripts/report</code><code class="bash plain">.sh</code></div>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<p>As the&nbsp;<em>crontab</em>&nbsp;isn’t the best user friendly tool to manage this kind of scheduling, it can be done on Jenkins with the following advantages&#8230;</p>
<ul>
<li>Very friendly interface to monitor their progress and current status</li>
<li>The logs are immediately availables and no need any special grant to access them</li>
<li>The job could be executed manually on Jenkins instead to have a scheduling</li>
<li>For some kind of jobs, no need to define users and passwords in plain text files as Jenkins do it in a secure way</li>
<li>The jobs could be defined as an API execution</li>
</ul>
<p>So, it could be a good solution to migrate the jobs related to PostgreSQL tasks to Jenkins instead of crontab.</p>
<p>On the other hand, most database administrators and developers have strong skills in scripting languages and it would be easy for them to develop small interfaces to deal with these scripts to implement the automated processes with the goal of improving their tasks. But remember, Jenkins most likely already has a set of functions to do it and these functionalities can make life easy for developers who choose to use them.</p>
<div class="pblock size_4">&nbsp;</div>
<p>Thus to define the execution of script it’s necessary to create a new job, selecting the “New Item” option.</p>
<div class="image">
<div class="image_inner"><img decoding="async" src="https://severalnines.com/sites/default/files/blog/node_5608/image16.jpg" alt="Figure 1 – &quot;New Item&quot; in order to define a job to execute a PostgreSQL script"></p>
<div class="description">Figure 1 – &#8220;New Item&#8221; in order to define a job to execute a PostgreSQL script</div>
</div>
</div>
<p>Then, after naming it, choose the type “FreeStyle projects” and click OK.</p>
<div class="image">
<div class="image_inner image_zoom"><img decoding="async" src="https://severalnines.com/sites/default/files/blog/node_5608/image2.jpg" alt="Figure 2 – Selection of the job (item) type"></p>
<div class="description">Figure 2 – Selection of the job (item) type</div>
</div>
</div>
<p>To finish the creation of this new job, in the section “Build” must be selected the option “Execute script” and in the command line box the path and parameterization of the script that will be executed:</p>
<div class="image">
<div class="image_inner"><img decoding="async" src="https://severalnines.com/sites/default/files/blog/node_5608/image7.png" alt="Figure 3 – Specification of the command to execute"></p>
<div class="description">Figure 3 – Specification of the command to execute</div>
</div>
</div>
<p>For this kind of job, it’s advisable to verify script permissions, because at least execution for the group the file belongs and for everyone must be set.</p>
<p>In this example, the script&nbsp;<em>query.sh</em>&nbsp;has read and execute permissions for everyone, read and execute permissions for the group and read write and execute for the user:</p>
<div>
<div id="highlighter_405244" class="syntaxhighlighter  bash">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td class="gutter">
<div class="line number1 index0 alt2">1</div>
<div class="line number2 index1 alt1">2</div>
<div class="line number3 index2 alt2">3</div>
</td>
<td class="code">
<div class="container">
<div class="line number1 index0 alt2"><code class="bash plain">slonik@severalnines:~</code><code class="bash plain">/db/scripts</code><code class="bash plain">$ </code><code class="bash functions">ls</code> <code class="bash plain">-l query.sh</code></div>
<div class="line number2 index1 alt1"><code class="bash plain">-rwxr-xr-x 1 slonik slonik 365 May 11 20:01 query.sh</code></div>
<div class="line number3 index2 alt2"><code class="bash plain">slonik@severalnines:~</code><code class="bash plain">/db/scripts</code><code class="bash plain">$</code></div>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<p>This script has a very simple set of statements, basically only calls to the utility psql in order to execute queries:</p>
<div>
<div id="highlighter_190569" class="syntaxhighlighter  bash">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td class="gutter">
<div class="line number1 index0 alt2">1</div>
<div class="line number2 index1 alt1">2</div>
<div class="line number3 index2 alt2">3</div>
<div class="line number4 index3 alt1">4</div>
<div class="line number5 index4 alt2">5</div>
<div class="line number6 index5 alt1">6</div>
<div class="line number7 index6 alt2">7</div>
</td>
<td class="code">
<div class="container">
<div class="line number1 index0 alt2"><code class="bash preprocessor bold">#!/bin/bash</code></div>
<div class="line number2 index1 alt1">&nbsp;</div>
<div class="line number3 index2 alt2"><code class="bash plain">/usr/lib/postgresql/10/bin/psql</code> <code class="bash plain">-U report -d db_deploy -c </code><code class="bash string">"select * from appl"</code> <code class="bash plain">&gt; </code><code class="bash plain">/home/slonik/db/scripts/appl</code><code class="bash plain">.dat</code></div>
<div class="line number4 index3 alt1"><code class="bash spaces">&nbsp;</code>&nbsp;</div>
<div class="line number5 index4 alt2"><code class="bash plain">/usr/lib/postgresql/10/bin/psql</code> <code class="bash plain">-U report -d db_deploy -c </code><code class="bash string">"select * from appl_users"</code> <code class="bash plain">&gt; </code><code class="bash plain">/home/slonik/db/scripts/appl_user</code><code class="bash plain">.dat</code></div>
<div class="line number6 index5 alt1"><code class="bash spaces">&nbsp;</code>&nbsp;</div>
<div class="line number7 index6 alt2"><code class="bash plain">/usr/lib/postgresql/10/bin/psql</code> <code class="bash plain">-U report -d db_deploy -c </code><code class="bash string">"select * from appl_rights"</code> <code class="bash plain">&gt; </code><code class="bash plain">/home/slonik/db/scripts/appl_rights</code><code class="bash plain">.dat</code></div>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<h2>Executing a PostgreSQL Process Composed of Three or More Scripts</h2>
<p>In this example, I’ll describe what you need to execute three different scripts in order to hide sensitive data and for that, we will follow the below steps&#8230;</p>
<ul>
<li>Import data from files</li>
<li>Prepare data to be masked</li>
<li>Backup of database with data masked</li>
</ul>
<p>So, to define this new job it’s necessary to select the option “New Item” in the Jenkins main page and then, after to assign a name, the “Pipeline” option must be chosen:</p>
<div class="image">
<div class="image_inner image_zoom"><img decoding="async" src="https://severalnines.com/sites/default/files/blog/node_5608/image9.jpg" alt="Figure 5 - Pipeline item in Jenkins"></p>
<div class="description">Figure 5 &#8211; Pipeline item in Jenkins</div>
</div>
</div>
<p>Once the job is saved in the “Pipeline” section, on the tab “Advanced project options”, the “Definition” field must be set to “Pipeline script”, as shown below:</p>
<div class="image">
<div class="image_inner image_zoom"><img decoding="async" src="https://severalnines.com/sites/default/files/blog/node_5608/image15.jpg" alt="Figure 6 - Groovy script in pipeline section"></p>
<div class="description">Figure 6 &#8211; Groovy script in pipeline section</div>
</div>
</div>
<p>As I mentioned at the beginning of the chapter, the used Groovy script it’s composed by three stages, it means three distinct parts (stages), as presented in the following script:</p>
<div>
<div id="highlighter_897752" class="syntaxhighlighter  java">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td class="gutter">
<div class="line number1 index0 alt2">1</div>
<div class="line number2 index1 alt1">2</div>
<div class="line number3 index2 alt2">3</div>
<div class="line number4 index3 alt1">4</div>
<div class="line number5 index4 alt2">5</div>
<div class="line number6 index5 alt1">6</div>
<div class="line number7 index6 alt2">7</div>
<div class="line number8 index7 alt1">8</div>
<div class="line number9 index8 alt2">9</div>
<div class="line number10 index9 alt1">10</div>
<div class="line number11 index10 alt2">11</div>
<div class="line number12 index11 alt1">12</div>
<div class="line number13 index12 alt2">13</div>
<div class="line number14 index13 alt1">14</div>
<div class="line number15 index14 alt2">15</div>
<div class="line number16 index15 alt1">16</div>
<div class="line number17 index16 alt2">17</div>
<div class="line number18 index17 alt1">18</div>
<div class="line number19 index18 alt2">19</div>
<div class="line number20 index19 alt1">20</div>
<div class="line number21 index20 alt2">21</div>
<div class="line number22 index21 alt1">22</div>
<div class="line number23 index22 alt2">23</div>
<div class="line number24 index23 alt1">24</div>
<div class="line number25 index24 alt2">25</div>
<div class="line number26 index25 alt1">26</div>
<div class="line number27 index26 alt2">27</div>
<div class="line number28 index27 alt1">28</div>
<div class="line number29 index28 alt2">29</div>
<div class="line number30 index29 alt1">30</div>
<div class="line number31 index30 alt2">31</div>
<div class="line number32 index31 alt1">32</div>
</td>
<td class="code">
<div class="container">
<div class="line number1 index0 alt2"><code class="java plain">def dir_live1=</code><code class="java string">'/data/ftp/server1'</code></div>
<div class="line number2 index1 alt1"><code class="java plain">def dir_live2=</code><code class="java string">'/data/ftp/server2'</code></div>
<div class="line number3 index2 alt2">&nbsp;</div>
<div class="line number4 index3 alt1"><code class="java plain">pipeline{</code></div>
<div class="line number5 index4 alt2"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">agent any</code></div>
<div class="line number6 index5 alt1"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">stages</code></div>
<div class="line number7 index6 alt2"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">{</code></div>
<div class="line number8 index7 alt1"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">stage(</code><code class="java string">'Import data from files'</code><code class="java plain">){</code></div>
<div class="line number9 index8 alt2"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">steps{</code></div>
<div class="line number10 index9 alt1"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">echo </code><code class="java string">"Import data from ${dir_live1} ..."</code></div>
<div class="line number11 index10 alt2"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">sh script :</code><code class="java string">"/home/slonik/db/scripts/import_data_from_files.sh ${dir_live1}"</code></div>
<div class="line number12 index11 alt1"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">echo </code><code class="java string">"Import data from $dir_live2 ..."</code></div>
<div class="line number13 index12 alt2"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">sh script :</code><code class="java string">"/home/slonik/db/scripts/import_data_from_files.sh ${dir_live2}"</code></div>
<div class="line number14 index13 alt1"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">}</code></div>
<div class="line number15 index14 alt2"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">}</code></div>
<div class="line number16 index15 alt1"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">stage(</code><code class="java string">'Prepare data to be masked'</code><code class="java plain">){</code></div>
<div class="line number17 index16 alt2"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">steps{</code></div>
<div class="line number18 index17 alt1"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">echo </code><code class="java string">"Purge data ..."</code></div>
<div class="line number19 index18 alt2"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">sh script :</code><code class="java string">"/home/slonik/db/scripts/purge_data.sh"</code></div>
<div class="line number20 index19 alt1"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">echo </code><code class="java string">"Mask data ..."</code></div>
<div class="line number21 index20 alt2"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">sh script :</code><code class="java string">"/home/slonik/db/scripts/mask_data.sh"</code></div>
<div class="line number22 index21 alt1"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">}</code></div>
<div class="line number23 index22 alt2"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">}</code></div>
<div class="line number24 index23 alt1"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">stage(</code><code class="java string">'Backup of database with data masked'</code><code class="java plain">){</code></div>
<div class="line number25 index24 alt2"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">steps{</code></div>
<div class="line number26 index25 alt1"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">echo </code><code class="java string">"Backup database after masking ..."</code></div>
<div class="line number27 index26 alt2"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">sh script :</code><code class="java string">"/home/slonik/db/scripts/backup_db.sh"</code></div>
<div class="line number28 index27 alt1"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">}</code></div>
<div class="line number29 index28 alt2"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">} </code></div>
<div class="line number30 index29 alt1"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>&nbsp;</div>
<div class="line number31 index30 alt2"><code class="java spaces">&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="java plain">}</code></div>
<div class="line number32 index31 alt1"><code class="java plain">}</code></div>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<p><a href="http://groovy-lang.org/" target="_blank" rel="nofollow noopener noreferrer">Groovy</a>&nbsp;is a Java-syntax-compatible object oriented programming language for the Java platform. It’s both a static and dynamic language with features similar to those of Python, Ruby, Perl and Smalltalk.</p>
<p>It’s easy to understand since this kind of script is based in a few statements…</p>
<h3>Stage</h3>
<p>Means the 3 processes that will be executed: “Import data from files”, “Prepare data to be masked”</p>
<p>and “Backup of database with data masked”.</p>
<h3>Step</h3>
<p>A “step” (often called a “build step”) is a single task that is part of a sequence. Each stage could be composed of several steps. In this example, the first stage has two steps.</p>
<div>
<div id="highlighter_759204" class="syntaxhighlighter  bash">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td class="gutter">
<div class="line number1 index0 alt2">1</div>
<div class="line number2 index1 alt1">2</div>
<div class="line number3 index2 alt2">3</div>
</td>
<td class="code">
<div class="container">
<div class="line number1 index0 alt2"><code class="bash plain">sh script :"</code><code class="bash plain">/home/slonik/db/scripts/import_data_from_files</code><code class="bash plain">.sh </code><code class="bash string">'/data/ftp/server1'</code></div>
<div class="line number2 index1 alt1">&nbsp;</div>
<div class="line number3 index2 alt2"><code class="bash plain">sh script :"</code><code class="bash plain">/home/slonik/db/scripts/import_data_from_files</code><code class="bash plain">.sh </code><code class="bash string">'/data/ftp/server2'</code></div>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="pblock size_4">&nbsp;</div>
<p>The data are being imported from two distinct sources.</p>
<p>In the previous example, it’s important to note that there are two variables defined at the beginning and with a global scope:</p>
<div>
<div id="highlighter_236971" class="syntaxhighlighter  bash">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td class="gutter">
<div class="line number1 index0 alt2">1</div>
<div class="line number2 index1 alt1">2</div>
</td>
<td class="code">
<div class="container">
<div class="line number1 index0 alt2"><code class="bash plain">dir_live1</code></div>
<div class="line number2 index1 alt1"><code class="bash plain">dir_live2</code></div>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<p>The scripts used in these three steps are calling the&nbsp;<em>psql</em>,&nbsp;<em>pg_restore</em>&nbsp;and&nbsp;<em>pg_dump</em>&nbsp;utilities.</p>
<p>Once the job is defined, it’s time to execute it and for that, it’s only necessary to click the option “Build Now”:</p>
<div class="image">
<div class="image_inner"><img decoding="async" src="https://severalnines.com/sites/default/files/blog/node_5608/image10.jpg" alt="Figure 7 – Execution job"></p>
<div class="description">Figure 7 – Execution job</div>
</div>
</div>
<p>After the build starts it’s possible to verify its progress.</p>
<div class="image">
<div class="image_inner image_zoom"><img decoding="async" src="https://severalnines.com/sites/default/files/blog/node_5608/image4.jpg" alt="Figure 8 – Starting “Build”"></p>
<div class="description">Figure 8 – Starting “Build”</div>
</div>
</div>
<p>The Pipeline Stage View plugin includes an extended visualization of Pipeline build history on the index page of a flow project under Stage View. This view is built as soon as the tasks are completed and each task is represented by column from the left to the right and it’s possible to view and compare the elapsed time for the serval executions (known as a Build in Jenkins).</p>
<p>Once the execution (also called a Build) finishes, it’s possible to get additional details, clicking on the finished thread (red box).</p>
<div class="image">
<div class="image_inner image_zoom"><img decoding="async" src="https://severalnines.com/sites/default/files/blog/node_5608/image13.jpg" alt="Figure 9 – Starting “Build”"></p>
<div class="description">Figure 9 – Starting “Build”</div>
</div>
</div>
<p>and then in “Console Output” option.</p>
<div class="image">
<div class="image_inner image_zoom"><img decoding="async" src="https://severalnines.com/sites/default/files/blog/node_5608/image14.jpg" alt="Figure 10 – Console output"></p>
<div class="description">Figure 10 – Console output</div>
</div>
</div>
<p>The previous views are extreme utility since they allow to have a perception of the runtime required of each stage.</p>
<p>Pipelines, also known as workflow, it’s a plugin that allows the definition of the application lifecycle and it’s a functionality used in Jenkins for Continuous delivery (CD).vThis plugin was built with requirements for a flexible, extensible and script-based CD workflow capability in mind.</p>
<p>This example is to hide sensitive data but for sure there are many other examples on a daily basis of PostgreSQL database administrator that can be executed on a pipeline job.</p>
<p>Pipeline has been available on Jenkins since version 2.0 and it’s an incredible solution!</p>
<div class="image">
<div class="image_inner"><img decoding="async" src="https://severalnines.com/sites/default/files/blog/node_5608/image12.jpg"></div>
</div>
<h2>Continuous Integration (CI) for PL/pgSQL Developments</h2>
<p>The continuous integration for the database development is not as easy as in other programming languages due to the data that can be lost, so it isn’t easy to keep the database in source control and deploy it on a dedicated server particularly once there are scripts that contain DDL (Data Definition Language) and DML (Data Manipulation Language) statements. This is because these kinds of statements modify the current state of the database and unlike other programming languages there is no source code to compile.</p>
<p>On the other hand, there are a set of database statements for which it’s possible the continuous integration as for other programming languages.</p>
<p>This example it’s based only in the development of procedures and it will illustrate the triggering of a set of tests (written in Python) by Jenkins once PostgreSQL scripts, on which are stored the code of the following functions, are committed in a code repository.</p>
<div class="image">
<div class="image_inner"><img decoding="async" src="https://severalnines.com/sites/default/files/blog/node_5608/image8.jpg" alt="Figure 11 – PLpg/SQL functions"></p>
<div class="description">Figure 11 – PLpg/SQL functions</div>
</div>
</div>
<p>These functions are simple and its content only have a few logic or a query in&nbsp;<em>PLpg/SQL</em>&nbsp;or&nbsp;<em>plperlu</em>&nbsp;language as the function&nbsp;<em>f_IsValidEmail</em>:</p>
<div>
<div id="highlighter_259774" class="syntaxhighlighter  sql">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td class="gutter">
<div class="line number1 index0 alt2">1</div>
<div class="line number2 index1 alt1">2</div>
<div class="line number3 index2 alt2">3</div>
<div class="line number4 index3 alt1">4</div>
<div class="line number5 index4 alt2">5</div>
<div class="line number6 index5 alt1">6</div>
<div class="line number7 index6 alt2">7</div>
</td>
<td class="code">
<div class="container">
<div class="line number1 index0 alt2"><code class="sql keyword">CREATE</code> <code class="sql color1">OR</code> <code class="sql color2">REPLACE</code> <code class="sql keyword">FUNCTION</code> <code class="sql plain">f_IsValidEmail(email text) </code><code class="sql keyword">RETURNS</code> <code class="sql plain">bool</code></div>
<div class="line number2 index1 alt1"><code class="sql plain">LANGUAGE plperlu</code></div>
<div class="line number3 index2 alt2"><code class="sql keyword">AS</code> <code class="sql plain">$$</code></div>
<div class="line number4 index3 alt1"><code class="sql plain">use Email::Address;</code></div>
<div class="line number5 index4 alt2"><code class="sql plain">my @addresses = Email::Address-&gt;parse($_[0]);</code></div>
<div class="line number6 index5 alt1"><code class="sql keyword">return</code> <code class="sql plain">scalar(@addresses) &gt; 0 ? 1 : 0;</code></div>
<div class="line number7 index6 alt2"><code class="sql plain">$$;</code></div>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<p>All the functions presented here do not depend on each other, and then there is no precedence either in its development or in its deployment. Also, as it will be verified ahead, there is no dependence on their validations.</p>
<p>So, in order to execute a set of validation scripts once a commit it’s performed in a code repository it’s necessary the creation of a build job (new item) in Jenkins:</p>
<div class="image">
<div class="image_inner"><img decoding="async" src="https://severalnines.com/sites/default/files/blog/node_5608/image6.png" alt="Figure 12 – &quot;Freestyle” project for Continuous Integration"></p>
<div class="description">Figure 12 – &#8220;Freestyle” project for Continuous Integration</div>
</div>
</div>
<p>This new build job should be created as “Freestyle” project and in the “Source code repository” section must be defined the repository URL and its credentials (orange box):</p>
<div class="image">
<div class="image_inner"><img decoding="async" src="https://severalnines.com/sites/default/files/blog/node_5608/image5.png" alt="Figure 13 – Source code repository"></p>
<div class="description">Figure 13 – Source code repository</div>
</div>
</div>
<p>In the section &#8220;Build Triggers&#8221; the option &#8220;GitHub hook trigger for GITScm polling&#8221; must be checked:</p>
<div class="image">
<div class="image_inner"><img decoding="async" src="https://severalnines.com/sites/default/files/blog/node_5608/image11.jpg" alt="Figure 14 – “Build triggers” section"></p>
<div class="description">Figure 14 – “Build triggers” section</div>
</div>
</div>
<p>Finally, in the “Build” section, the option “Execute Shell” must be selected and in the command box the scripts that will do the validation of the developed functions:</p>
<div class="image">
<div class="image_inner image_zoom"><img decoding="async" src="https://severalnines.com/sites/default/files/blog/node_5608/image3.jpg" alt="Figure 15 – “Build environment” section"></p>
<div class="description">Figure 15 – “Build environment” section</div>
</div>
</div>
<div class="pblock size_4">&nbsp;</div>
<p>The purpose is to have one validation script for each developed function.</p>
<p>This Python script has a simple set of statements that will call these procedures from a database with some predefined expected results:</p>
<div>
<div id="highlighter_163633" class="syntaxhighlighter  python">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td class="gutter">
<div class="line number1 index0 alt2">1</div>
<div class="line number2 index1 alt1">2</div>
<div class="line number3 index2 alt2">3</div>
<div class="line number4 index3 alt1">4</div>
<div class="line number5 index4 alt2">5</div>
<div class="line number6 index5 alt1">6</div>
<div class="line number7 index6 alt2">7</div>
<div class="line number8 index7 alt1">8</div>
<div class="line number9 index8 alt2">9</div>
<div class="line number10 index9 alt1">10</div>
<div class="line number11 index10 alt2">11</div>
<div class="line number12 index11 alt1">12</div>
<div class="line number13 index12 alt2">13</div>
<div class="line number14 index13 alt1">14</div>
<div class="line number15 index14 alt2">15</div>
<div class="line number16 index15 alt1">16</div>
<div class="line number17 index16 alt2">17</div>
<div class="line number18 index17 alt1">18</div>
<div class="line number19 index18 alt2">19</div>
<div class="line number20 index19 alt1">20</div>
<div class="line number21 index20 alt2">21</div>
<div class="line number22 index21 alt1">22</div>
<div class="line number23 index22 alt2">23</div>
<div class="line number24 index23 alt1">24</div>
</td>
<td class="code">
<div class="container">
<div class="line number1 index0 alt2"><code class="python comments">#!/usr/bin/python</code></div>
<div class="line number2 index1 alt1"><code class="python keyword">import</code> <code class="python plain">psycopg2</code></div>
<div class="line number3 index2 alt2">&nbsp;</div>
<div class="line number4 index3 alt1"><code class="python plain">con </code><code class="python keyword">=</code> <code class="python plain">psycopg2.connect(database</code><code class="python keyword">=</code><code class="python string">"db_deploy"</code><code class="python plain">, user</code><code class="python keyword">=</code><code class="python string">"postgres"</code><code class="python plain">, password</code><code class="python keyword">=</code><code class="python string">"postgres10"</code><code class="python plain">, host</code><code class="python keyword">=</code><code class="python string">"localhost"</code><code class="python plain">, port</code><code class="python keyword">=</code><code class="python string">"5432"</code><code class="python plain">)</code></div>
<div class="line number5 index4 alt2"><code class="python plain">cur </code><code class="python keyword">=</code> <code class="python plain">con.cursor() </code></div>
<div class="line number6 index5 alt1">&nbsp;</div>
<div class="line number7 index6 alt2"><code class="python plain">email_list </code><code class="python keyword">=</code> <code class="python plain">{ </code><code class="python string">'max1450@mail.com'</code> <code class="python plain">: </code><code class="python color1">True</code><code class="python plain">,</code></div>
<div class="line number8 index7 alt1"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python string">'tintinmail.com'</code>&nbsp;&nbsp; <code class="python plain">:&nbsp; </code><code class="python color1">False</code><code class="python plain">,</code></div>
<div class="line number9 index8 alt2"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python string">'.@severalnines'</code>&nbsp;&nbsp; <code class="python plain">: </code><code class="python color1">False</code><code class="python plain">,</code></div>
<div class="line number10 index9 alt1"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python string">'director#mail.com'</code><code class="python plain">: </code><code class="python color1">False</code><code class="python plain">,</code></div>
<div class="line number11 index10 alt2"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python string">'1942@xmail.com'</code>&nbsp;&nbsp; <code class="python plain">: </code><code class="python color1">True</code></div>
<div class="line number12 index11 alt1"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">}</code></div>
<div class="line number13 index12 alt2"><code class="python plain">result_msg</code><code class="python keyword">=</code> <code class="python string">"f_IsValidEmail -&gt; OK"</code></div>
<div class="line number14 index13 alt1">&nbsp;</div>
<div class="line number15 index14 alt2"><code class="python keyword">for</code> <code class="python plain">key </code><code class="python keyword">in</code> <code class="python plain">email_list:</code></div>
<div class="line number16 index15 alt1"><code class="python spaces">&nbsp;&nbsp;</code><code class="python plain">cur.callproc(</code><code class="python string">'f_IsValidEmail'</code><code class="python plain">, (key,))</code></div>
<div class="line number17 index16 alt2"><code class="python spaces">&nbsp;&nbsp;</code><code class="python plain">row </code><code class="python keyword">=</code> <code class="python plain">cur.fetchone()</code></div>
<div class="line number18 index17 alt1"><code class="python spaces">&nbsp;&nbsp;</code><code class="python keyword">if</code> <code class="python plain">email_list[key]!</code><code class="python keyword">=</code><code class="python plain">row[</code><code class="python value">0</code><code class="python plain">]:</code></div>
<div class="line number19 index18 alt2"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">result_msg</code><code class="python keyword">=</code> <code class="python string">"f_IsValidEmail -&gt; Nok"</code></div>
<div class="line number20 index19 alt1">&nbsp;</div>
<div class="line number21 index20 alt2"><code class="python functions">print</code> <code class="python plain">result_msg</code></div>
<div class="line number22 index21 alt1">&nbsp;</div>
<div class="line number23 index22 alt2"><code class="python plain">cur.close()</code></div>
<div class="line number24 index23 alt1"><code class="python plain">con.close()</code></div>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<p>This script will test the presented&nbsp;<em>PLpg/SQL</em>&nbsp;or&nbsp;<em>plperlu</em>&nbsp;functions and it will be executed after each commit in the code repository in order to avoid regressions on the developments.</p>
<p>Once this job build is executed, the log executions could be verified.</p>
<div class="image">
<div class="image_inner image_zoom"><img decoding="async" src="https://severalnines.com/sites/default/files/blog/node_5608/image1.jpg" alt="Figure 16 – “Console Output”"></p>
<div class="description">Figure 16 – “Console Output”</div>
</div>
</div>
<p>This option presents the final status: SUCCESS or FAILURE, the workspace, the executed files/script, the created temporary files and the error messages (for the failure ones)!</p>
<h2>Conclusion</h2>
<p>In summary, Jenkins is known as a great tool for Continuous Integration (CI) and Continuous Delivery (CD), however, it can be used for various functionalities like,</p>
<ul>
<li>Scheduling tasks</li>
<li>Execution of scripts</li>
<li>Monitoring Processes</li>
</ul>
<p>For all of these purposes on each execution (Build on Jenkins vocabulary) it can be analyzed the logs and elapsed time.</p>
<p>Due to a large number of available plugins it could avoid some developments with a specific aim, probably there is a plugin that does exactly what you’re looking for, it’s just a matter of searching the&nbsp;<a href="https://updates.jenkins.io/" target="_blank" rel="nofollow noopener noreferrer">update center</a>&nbsp;or Manage Jenkins&gt;&gt;Manage Plugins inside the web application.</p>
<p>The post <a href="http://kostacipo.stream/automating-postgresql-daily-tasks-using-jenkins/">Automating PostgreSQL Daily Tasks Using Jenkins</a> appeared first on <a href="http://kostacipo.stream">Tech Chronicles</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>http://kostacipo.stream/automating-postgresql-daily-tasks-using-jenkins/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>Creating a Secure Pipeline: Jenkins with SonarQube and DependencyCheck</title>
		<link>http://kostacipo.stream/creating-a-secure-pipeline-jenkins-with-sonarqube-and-dependencycheck/</link>
					<comments>http://kostacipo.stream/creating-a-secure-pipeline-jenkins-with-sonarqube-and-dependencycheck/#respond</comments>
		
		<dc:creator><![CDATA[Majordomo]]></dc:creator>
		<pubDate>Wed, 08 Jan 2020 10:45:34 +0000</pubDate>
				<category><![CDATA[DevOps]]></category>
		<category><![CDATA[General]]></category>
		<category><![CDATA[Software Development]]></category>
		<category><![CDATA[ci/cd]]></category>
		<category><![CDATA[jenkins]]></category>
		<category><![CDATA[owasp]]></category>
		<category><![CDATA[sonarqube]]></category>
		<guid isPermaLink="false">http://www.kostacipo.stream/?p=1639</guid>

					<description><![CDATA[<p>&#160; Configuring Jenkins To Build WebGoat We&#8217;re going to scan a known vulnerable webapp, WebGoat, which is an OWASP project used for learning basic web penetration testing skills and vulnerabilities. A good scanner should find a lot of things! A quick note: We were initially going to use Mutillidae, another vulnerable app written in PHP. [&#8230;]</p>
<p>The post <a href="http://kostacipo.stream/creating-a-secure-pipeline-jenkins-with-sonarqube-and-dependencycheck/">Creating a Secure Pipeline: Jenkins with SonarQube and DependencyCheck</a> appeared first on <a href="http://kostacipo.stream">Tech Chronicles</a>.</p>
]]></description>
										<content:encoded><![CDATA[<p>&nbsp;</p>
<h2 id="configuring-jenkins-to-build-webgoat">Configuring Jenkins To Build WebGoat</h2>
<p>We&#8217;re going to scan a known vulnerable webapp, <a href="https://webgoat.github.io/WebGoat/">WebGoat</a>, which is an OWASP project used for learning basic web penetration testing skills and vulnerabilities. A good scanner should find a lot of things!</p>
<p>A quick note: We were initially going to use <a href="https://github.com/webpwnized/mutillidae">Mutillidae</a>, another vulnerable app written in PHP. However we couldn&#8217;t find any good open source PHP Static analyzers that would catch the vulnerabilities. </p>
<p>Anyway, let&#8217;s get on with Jenkins. Navigate in your browser to <a href="http://localhost:8080">http://localhost:8080</a> and enter the admin password shown in the terminal running docker. Go ahead and install the default plugins (for a deployed instance, I would recommend only installing plugins you will actually use) and create your first admin user.</p>
<p>WebGoat requires Java 11 to build, which Jenkins won&#8217;t install automatically. Head over to the main page -&gt; Manage Jenkins -&gt; Global Tool Configuration. There are two sections here we will update now: JDK installation and Maven installations. We need to add a link to a Java 11 installer &#8211; we used <a href="https://download.java.net/java/GA/jdk11/13/GPL/openjdk-11.0.1_linux-x64_bin.tar.gz">https://download.java.net/java/GA/jdk11/13/GPL/openjdk-11.0.1_linux-x64_bin.tar.gz</a> and we can use the default maven. Your config should look like this:</p>
<figure class="kg-card kg-image-card kg-card-hascaption"><img decoding="async" class="kg-image" src="https://nullsweep.com/content/images/2019/05/jenkins_jdk_install_config.png"><figcaption>Jenkins JDK11 Install configuration</figcaption></figure>
<figure class="kg-card kg-image-card kg-card-hascaption"><img decoding="async" class="kg-image" src="https://nullsweep.com/content/images/2019/05/jenkins_maven_install_config.png"><figcaption>Jenkins Maven install configuration</figcaption></figure>
<p>We sometimes see Jenkins have trouble installing a JDK this way if more than one JDK is installed in the system. If this is the first one, there should be no problems.</p>
<p>Finally, we have to set the JAVA_HOME variable. In the Jenkins -&gt; Manage Jenkins -&gt; Configure System menu, enable environment varaibles and set JAVA_HOME equal to /var/jenkins_home/tools/hudson.model.JDK/openjdk11/jdk-11.0.1/.</p>
<p>Now let&#8217;s create a pipeline for WebGoat and make sure it builds successfully. Back on the main page choose new item -&gt; freestyle project.</p>
<p>The initial setup is pretty simple:</p>
<ul>
<li>Add Webgoat to the various github setting locations (https://github.com/WebGoat/WebGoat/)</li>
<li>Set the target branch to */develop</li>
<li>Create a maven build step (&#8220;Invoke top level maven targets&#8221;) and give it the command &#8220;clean install&#8221;</li>
</ul>
<p>Here is the full pipeline configuration:</p>
<figure class="kg-card kg-image-card kg-card-hascaption"><img decoding="async" class="kg-image" src="https://nullsweep.com/content/images/2019/05/initial_webgoat_pipeline.png"><figcaption>Basic WebGoat Pipeline</figcaption></figure>
<p>Try running it and making sure that everything builds successfully.</p>
<h2 id="adding-sonarqube-and-dependencycheck">Adding SonarQube and DependencyCheck</h2>
<h3 id="sonarqube-setup-security">SonarQube setup &amp; security</h3>
<p>We already have a SonarQube instance running, we just need to link and configure Jenkins to use it. Log in to <a href="http://localhost:9000">http://localhost:9000</a> and use the default sonarQube login of admin/admin.</p>
<p>Although this is only for practice, I still want to secure our SonarQube instance, so do the following:</p>
<ul>
<li>Change the admin password</li>
<li>Go to administration-&gt; security and turn on &#8220;Force user authentication&#8221;</li>
<li>Create a new user for Jenkins.</li>
<li>Log into the new user, go to the profile -&gt; security section, and generate a token. Copy this for later use.</li>
</ul>
<p>Finally, create a project named &#8220;webgoat&#8221; with your jenkins user.</p>
<h3 id="configure-the-plugins-for-jenkins">Configure the plugins for Jenkins</h3>
<p>We will need two new plugins for jenkins. In the Jenkins home page, go to Mange Jenkins -&gt; Manage Plugins. On the Available tab find and select &#8220;OWASP Dependency-Check Plugin&#8221; and &#8220;SonarQube Scanner for Jenkins&#8221;. Install them without restarting.</p>
<p>Back on the Jenkins home, go to Manage Jenkins -&gt; Global Tool Configuration. You should see a new option for SonarQube Scanner. Add an installation here (I just chose the latest from Maven Central) and save.</p>
<p>Finally, head over to Jenkins -&gt; Manage Jenkins -&gt; Configure System and add a sonarqube instance. The URL with our docker container is http://sonarqube:9000 and the token should be the one you saved while setting up the Jenkins user in SonarQube. Here is my setup:</p>
<figure class="kg-card kg-image-card kg-card-hascaption"><img decoding="async" class="kg-image" src="https://nullsweep.com/content/images/2019/05/sonarqube_settings.png"><figcaption>SonarQube Settings</figcaption></figure>
<p>One other thing I had to do to get SonarQube working properly. For some reason I couldn&#8217;t completely determine, the SonarQube startup script was truncating the JAVA_HOME path incorrectly, causing errors during the pipeline. To solve this, log into the docker container manually and update the sonar script to the proper JAVA_HOME.</p>
<pre class=" language-bash"><code class=" language-bash">$ docker <span class="token function">exec</span> -it secure_pipeline_jenkins_1 <span class="token function">bash</span>
jenkins@2ea0acb5905d:/$ <span class="token function">cd</span> /var/jenkins_home/tools/hudson.plugins.sonar.SonarRunnerInstallation/sonarqube
jenkins@2ea0acb5905d:~/tools/hudson.plugins.sonar.SonarRunnerInstallation/sonarqube$ <span class="token function">head</span> bin/sonar-scanner
<span class="token comment">#!/bin/sh</span>
<span class="token comment">#</span>
<span class="token comment"># SonarQube Scanner Startup Script for Unix</span>
<span class="token comment">#</span>
<span class="token comment"># Optional ENV vars:</span>
<span class="token comment">#   SONAR_SCANNER_OPTS - Parameters passed to the Java VM when running the SonarQube Scanner</span>
<span class="token comment">#   SONAR_SCANNER_DEBUG_OPTS - Extra parameters passed to the Java VM for debugging</span>
<span class="token comment">#   JAVA_HOME - Location of Java's installation</span>

JAVA_HOME<span class="token operator">=</span><span class="token string">"/var/jenkins_home/tools/hudson.model.JDK/openjdk11-remote/jdk-11.0.1"</span>
</code></pre>
<h3 id="add-sonarqube-and-dependencycheck-to-the-pipeline">Add SonarQube and DependencyCheck to the pipeline</h3>
<p>Now we can add these to our pipeline and start scanning with every build.</p>
<p>In the pipeline created earlier, add two new build steps &#8211; Invoke Dependency-Check analysis and Execute SonarQube Scanner. In the SonarQube scanner, add the configuration settings required &#8211; the project key and name should match the project you created in SonarQube.</p>
<pre class=" language-bash"><code class=" language-bash">sonar.projectKey<span class="token operator">=</span>webgoat
sonar.projectName<span class="token operator">=</span>webgoat
sonar.projectVersion<span class="token operator">=</span>1.0
sonar.language<span class="token operator">=</span>java
sonar.java.binaries<span class="token operator">=</span>**/target/classes
sonar.exclusions<span class="token operator">=</span>**/*.ts
</code></pre>
<p>I am excluding the TypeScript files above since we did not setup Node or a JS build step for our project. In a real project, we would want to ensure that they were also scannable.</p>
<p>In the DependencyCheck advanced section, check to generate HTML reports as well for easier viewing.</p>
<p>Here is my full pipeline configuration now:</p>
<figure class="kg-card kg-image-card kg-card-hascaption"><img decoding="async" class="kg-image" src="https://nullsweep.com/content/images/2019/05/webgoat_pipeline_with_sast.png"><figcaption>Full Secure Pipeline</figcaption></figure>
<p>Kick off a build and make sure it runs correctly. Afterwards, you should be able to see results.</p>
<h3 id="viewing-reports">Viewing Reports</h3>
<p>If all runs successfully, logging into SonarQube will show you security scan details (with plenty of findings!) and the pipeline can show you the dependencyCheck results in the workspace -&gt; dependency-check-report.html file.</p>
<figure class="kg-card kg-image-card kg-card-hascaption"><img decoding="async" class="kg-image" src="https://nullsweep.com/content/images/2019/05/dependencyCheck_results.png"><figcaption>DependencyCheck Sample Data</figcaption></figure>
<figure class="kg-card kg-image-card kg-card-hascaption"><img decoding="async" class="kg-image" src="https://nullsweep.com/content/images/2019/05/webgoat_sonarqube_report.png"><figcaption>SonarQube Findings</figcaption></figure>
<p>You can and should at this point consider additional SonarQube plugins (or other SAST tools) that are specifically for your languages and frameworks.</p>
<h2 id="breaking-the-build">Breaking the Build</h2>
<p>We want to know when something isn&#8217;t working right at the build phase. SonarQube gives us this for free with the plugin (you should see a nice red ERROR tag under the SonarQube Quality gate) but DependencyCheck requires one more configuration.</p>
<p>Add a post-build check for &#8220;Publish Dependency Check Results&#8221; and expand the advanced tabs. Just add some threshold data and the build will fail or be marked unstable according to the rules set.</p>
<p>Here &nbsp;is our final pipeline configuration, fully expanded.</p>
<figure class="kg-card kg-image-card"><img decoding="async" class="kg-image" src="https://nullsweep.com/content/images/2019/05/jenkins_full_sast.png"></figure>
<h2 id="final-thoughts">Final Thoughts</h2>
<p>Getting a CI/CD pipeline running with some basic security checks can be done within a few minutes. This will help keep your published artifacts in better shape and ensure the team has an opportunity to learn about security issues as soon as they emerge.</p>
<p>The post <a href="http://kostacipo.stream/creating-a-secure-pipeline-jenkins-with-sonarqube-and-dependencycheck/">Creating a Secure Pipeline: Jenkins with SonarQube and DependencyCheck</a> appeared first on <a href="http://kostacipo.stream">Tech Chronicles</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>http://kostacipo.stream/creating-a-secure-pipeline-jenkins-with-sonarqube-and-dependencycheck/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>Setup SonarQube in OpenShift for scanning projects via Jenkins</title>
		<link>http://kostacipo.stream/setup-sonarqube-in-openshift-for-scanning-projects-via-jenkins/</link>
					<comments>http://kostacipo.stream/setup-sonarqube-in-openshift-for-scanning-projects-via-jenkins/#respond</comments>
		
		<dc:creator><![CDATA[Majordomo]]></dc:creator>
		<pubDate>Tue, 03 Dec 2019 09:30:25 +0000</pubDate>
				<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Tools]]></category>
		<category><![CDATA[jenkins]]></category>
		<category><![CDATA[openshift]]></category>
		<category><![CDATA[sonarqube]]></category>
		<guid isPermaLink="false">http://www.kostacipo.stream/?p=1403</guid>

					<description><![CDATA[<p>&#160; When you have software development projects that you are building and running on Red Hat OpenShift you are probably testing them out locally with Minishift, getting the deployment correct, testing source-to-image or running your Dockerfile builds with environment variables to ensure your CI/CD pipeline is ready to roll. Or maybe you are testing out [&#8230;]</p>
<p>The post <a href="http://kostacipo.stream/setup-sonarqube-in-openshift-for-scanning-projects-via-jenkins/">Setup SonarQube in OpenShift for scanning projects via Jenkins</a> appeared first on <a href="http://kostacipo.stream">Tech Chronicles</a>.</p>
]]></description>
										<content:encoded><![CDATA[<p>&nbsp;</p>
<p>When you have software development projects that you are building and running on Red Hat OpenShift you are probably testing them out locally with Minishift, getting the deployment correct, testing source-to-image or running your Dockerfile builds with environment variables to ensure your CI/CD pipeline is ready to roll. Or maybe you are testing out platforms or containers and kubernetes and you want to get a good developer experience when doing it. You also could be testing out this DevOps thing people have been talking about for years and years.</p>
<p>One other thing to remember is the “Sec” part of DevSecOps: Sec = security. In my case, I use&nbsp;SonarQube locally and on my platform as part of my “Sec” steps to scan my projects and look for errors, vulnerabilities, bad coding practices, and the like. This post is to show how to setup SonarQube via a template within OpenShift (and Minishift) and call it from your Jenkins pipeline.</p>
<h4>Setup Minishift</h4>
<p>First things first, you need Minishift on your machine. I use a Mac for all my development so I used&nbsp;brew&nbsp;to install the latest Minishift executable. You can see other ways to setup Minishift&nbsp;here. I also use&nbsp;VirtualBox&nbsp;for my virtual machine driver for Minishift. I have a script that runs this for doing all my local Minishift development work that adds the vm-driver for VirtualBox, memory of 10GB, and disk-size of 40GB. I also run with the profile flag and save off the VM under a specific name so I can keep track of it. All personal preferences of course and you can run&nbsp;minishift -h&nbsp;to see the options. Figure out the options you wish to use and run minishift start with those options. It will pull down images, start the system, and give you information on how to log in. When that is up and running we are ready to setup SonarQube.</p>
<section class="cu cv cw cx cy">
<div class="n p">
<div class="ac ae af ag ah cz aj ak">
<div class="ho hp dr hq ak">
<div class="cl cm hh">
<div class="hw r dr hx">
<div class="hy r"><img loading="lazy" decoding="async" class="ly ns cp t u ht ak ic aligncenter" role="presentation" src="https://miro.medium.com/max/1080/1*4uS8R-v3dNo25Z_NIo_f7w.png" width="1080" height="20"></div>
</div>
</div>
</div>
<p style="text-align: center;" data-selectable-paragraph="">Minishift Command</p>
<h4>Optional: Enable the Admin-User AddOn</h4>
<p>This is optional but I like doing this locally. I enable the admin-user addon for Minishift. This lets you log in with ‘god’ rights. You can run&nbsp;minishift addon enable admin-user&nbsp;and then run&nbsp;minishift addon apply admin-user&nbsp;to do this. Once enabled and then applied you can login with admin/admin and see all the inner workings of Minishift and get a glimpse of what OpenShift has under the covers. There are other addons you can see by running the command&nbsp;minishift addons list&nbsp;as well. (Che is the other one I am testing out.)</p>
<h4>Setup the SonarQube Project</h4>
<p>Now that we have Minishift setup, you can go to the URL that was listed when you started it in the terminal window and login with admin/admin. Click the Create Project button on the top right and enter&nbsp;sonarqube&nbsp;(all lower case) for the Name field and&nbsp;SonarQube&nbsp;for the Display Name field. You can enter a description if you like to ensure you know what it is later when you look at the project listings.</p>
<div class="ho hp dr hq ak">
<div class="cl cm im">
<div class="hw r dr hx">
<div class="in r"><img loading="lazy" decoding="async" class="ly ns cp t u ht ak ic" role="presentation" src="https://miro.medium.com/max/1405/1*UpWhGpXocWCSrpAqQbbf2g.png" width="1405" height="878"></div>
</div>
</div>
</div>
<p style="text-align: center;" data-selectable-paragraph="">Add the SonarQube project on Minishift</p>
<p>Click the Create button and then click the SonarQube link that is created for you in the project listings. You will get to the blank screen for a Minishift project. From here you can deploy images, import deployment settings (we are doing this one), import other definitions in YAML format as well as select from templates in the catalog. I encourage you to see how you can&nbsp;import other templates, create your own templates for faster deployments, and get more familiar with this interface if you are going the OpenShift way.</p>
<p>I was inspired by the&nbsp;OpenShift Demos GH repo on SonarQube&nbsp;however I needed to update my image and persist data differently. So I went a different route. Not better or worse just different.</p>
<p>Now that you are in the SonarQube project click the “Import YAML / JSON” button and copy and paste&nbsp;this deployment from this GitHub repo. Click the Create button on the bottom right and ensure “Process the Template” is checked. Click Continue to get a listing of parameters. I will go into more detail later on these settings of saving data for the extensions (plugins), logs, temp space as well as the PostgreSQL database space.</p>
<p>You can adjust the space if you wish or just leave the defaults. I usually leave the database password field blank and it generates its own when I am doing local development.</p>
<p>When all is ready click the Create button and then the Close button and you will see a&nbsp;BUNCH&nbsp;of things were created. (Note the default SonarQube login is also admin/admin. If you keep this up you will want to change it OR link in oAuth, GitHub login, or Keycloak.)</p>
<p>What did we just do? Well we create a pod with a PostgreSQL image that when run exposes port 5432 internally to the project. We created a pod with a SonarQube 7.4-community image that when run exposes port 9000 through the https://sonarqube-sonarqube.x.x.x.x route. We have a database connection from SonarQube to PostgreSQL. And we have 5 areas of storage setup to persist data for these containers that are run: database data, SonarQube data, SonarQube extensions/plugins, SonarQube logs, and SonarQube temp space.</p>
<p>Do we need all of these? Maybe not if you want to write into the container and not keep it if it restarts. I do not like doing that. So I do it this way. Feel free to use and adjust to your will.</p>
<div class="ho hp dr hq ak">
<div class="cl cm ip">
<div class="hw r dr hx">
<div class="iq r"><img loading="lazy" decoding="async" class="ly ns cp t u ht ak ic" role="presentation" src="https://miro.medium.com/max/1406/1*R0LhdFVHPAJC7uYoYL_mCg.png" width="1406" height="874"></div>
</div>
</div>
</div>
<p style="text-align: center;" data-selectable-paragraph="">SonarQube 7.4 Community Edition setup in Minishift</p>
<h4>Let’s Dive In!</h4>
<p>To get more familiar with what our&nbsp;YAML&nbsp;file actually did we need to explore a little. We setup 2 deployments inside our project, one for each application. One application is SonarQube itself and we are using the image from&nbsp;DockerHub&nbsp;for that. The other application is PostgreSQL and OpenShift comes with that image already. So we just used it. If you click the Applications menu on the left and then Deployments you can see the 2 deployments we have. Click further on the links to see what each one has setup. For each deployment OpenShift will list #1, #2, #3 for the number of the deployment you have running. We just did this so you should see the #1 hyperlinked. Click on it to get something like the below image.</p>
<p>What you can see from the highlighted region are those storage listings, persistent volumes and persistent volume claims in Minishift language. This is telling OpenShift that for those areas of my container running (i.e. /opt/sonarqube/extensions) I want you to save that data in OpenShift to persist it. And as the container comes up or is redeployed mount this path inside the container for reuse. I am simplifying it however that is the gist of what this is doing. You also can see these Mounts going against the Storage listing under the Storage menu on the left. I tried to match the names to have them make sense for you.</p>
<h4>Log into SonarQube</h4>
<p>Go back to the Overview section of your project and you will see the&nbsp;https://sonarqube-sonarqube.x.x.x&nbsp;listing. This is the Route exposing a URL to the internal SonarQube container running. We only have a route for SonarQube. We do not need an external route for PostgreSQL as only SonarQube has to talk to it. Click that link and log in with admin/admin to ensure you can get to SonarQube. If it starts a tutorial click Skip this tutorial. You should get into SonarQube and see a screen like below. Now get to work!</p>
<h4>Use Jenkins in projects to scan code with SonarQube</h4>
<p>Having SonarQube is great and all&nbsp;only if you use it. I have an&nbsp;example microservice API project&nbsp;in dotnet core 2.1 that you could put into a brand new project and then use its&nbsp;deployment file&nbsp;to deploy 3 pods: Jenkins, the database, and the API. Then log into Jenkins, approve the OpenShift login, and see how Jenkins works within OpenShift. For SonarQube to work you need to add the SonarQube plugin on the Manage Jenkins → Manage Plugins page and restart. Then go into the Manage Jenkins → Configure System and setup the SonarQube plugin with the name if the server (i.e. SonarQube-Server), the internal-to-Openshift URL (i.e.&nbsp;http://sonarqube.sonarqube.svc), and a generated token from your account page so you can push scan results from the Jenkins process. You also could fork this project linked above and play around to get more familiar. The best way to learn is to do!</p>
<p>The steps on how to use SonarQube with Jenkins are&nbsp;here&nbsp;and are also all over Google.com and StackOverflow.com as well. To use that project linked above you have to load up some Jenkins slave images and update your Jenkinsfile to use them.</p>
<h4>Advanced: .NET Core Sonar Scanner Jenkins Slave</h4>
<p>There is a .Net Core SonarQube Scanner Jenkins slave image on this other repo&nbsp;that you can import by using the YAML files and importing into the ‘openshift’ project in OpenShift. You can use&nbsp;this project here to create in OpenShift to test it out. The Jenkins slave setup in the Jenkins Configure System screen will need to match the name of the labels in the Jenkinsfile. For now get used to SonarQube and see what you can do!</p>
</div>
</div>
</section>
<p>The post <a href="http://kostacipo.stream/setup-sonarqube-in-openshift-for-scanning-projects-via-jenkins/">Setup SonarQube in OpenShift for scanning projects via Jenkins</a> appeared first on <a href="http://kostacipo.stream">Tech Chronicles</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>http://kostacipo.stream/setup-sonarqube-in-openshift-for-scanning-projects-via-jenkins/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
	</channel>
</rss>
