tag:blogger.com,1999:blog-195172922024-03-19T09:48:23.465+01:00One Jar To Rule Them AllAlexhttp://www.blogger.com/profile/11632964711752480304noreply@blogger.comBlogger186125tag:blogger.com,1999:blog-19517292.post-27807393175203550202019-01-30T16:13:00.001+01:002019-01-30T16:13:14.194+01:00DRY with Kubernetes Operator Framework<div class="separator" style="clear: both; text-align: center;">
<a href="https://avatars3.githubusercontent.com/u/38202270?s=280&v=4" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="280" data-original-width="280" height="200" src="https://avatars3.githubusercontent.com/u/38202270?s=280&v=4" width="200" /></a></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: justify;">
<b><u>Introduction:</u></b><br />
<br />
An <b>Operator</b> is a method of packaging, deploying and managing a <b>Kubernetes</b> application. To make it, you use Kubernetes API to generate and deploy the application. You can think about a way of extending Kubernetes resources to be custom to your needs. This enables us to not have to repeat the same resource configuration every time but just the things that are different.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
An example of an operator might be the deployment of service of your microservice architecture. For your service, you need to create a deployment file where you specify a lot of parameters such as the container, the deployment information, environment variables to configure the service, liveness and readiness probes, ... Then when you need to release a new version of this service, you just take this deployment file, you specify everything again but with one small change (the version part of docker image is updated). So everything is exactly the same except one field (the version part) and you apply the resource to <b>Kubernetes</b> cluster. Why you need to repeat everything every time you want to release a new version when the only important thing is the version number?</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
An <b>Operator</b> let you fix this. You specify all common part as a <i>custom resource </i>(write once) and then for every new version of the service you only need to create a new resource of kind <i>custom resource</i>, with only the uncommon part set, in our example the version number.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
In a big simplification (I repeat a big simplification) you can think about <b>Operators</b> as a way to create a template with some dynamic values that are set in creation time. The biggest difference with a template is that the common content ("template") is created programmatically by the operator so you've got the freedom to change the resources dynamically.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Apart from that, with an <b>Operator</b>, you use Kubernetes API to decide when and how to deploy each of the resources.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<div style="text-align: justify;">
Let's start with a really simple example which might help you understand how powerful is <b>Operators</b> and why you should start using them.</div>
</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<div style="text-align: justify;">
Suppose that I have one simple service which prints to console a message. This message is set in the command line section. So the resource file to deploy this service might look like:</div>
</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<script src="https://gist.github.com/lordofthejars/71567ae565620e5c7764c737b7c6b72a.js"></script>
</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<div style="text-align: justify;">
As you can see it is fairly simple, but what's happen if now you want to deploy a new version of the service which instead of printing "Hello Alex", it prints "Hello Soto"? Well, you just need to create a new file which is exactly the same but changing the command line part. But instead of doing this, let's create an operator where you only need to specify the message to print, and the release of the service "just happens".</div>
<br />
<u><b>What you need:</b></u><br />
<br /></div>
<div style="text-align: left;">
<div style="text-align: justify;">
To create an operator, for this guide you need:</div>
<ul>
<li style="text-align: justify;">Go (<a href="https://golang.org/">https://golang.org/</a>)</li>
<li style="box-sizing: border-box; text-align: justify;">dep version v0.5.0+.</li>
<li style="text-align: justify;">Operators-SDK installed (for this post version 0.3.0 has been used) (<a href="https://github.com/operator-framework/operator-sdk/releases">https://github.com/operator-framework/operator-sdk/releases</a>)</li>
<li style="text-align: justify;">Kubernetes cluster v.1.11.0+. For my example, I am using Minishift + OpenShift 3.11 but might work with any Kuberntes implementation.</li>
<li style="text-align: justify;">kubectl/oc (for OpenShift <a href="https://github.com/openshift/origin/releases">https://github.com/openshift/origin/releases</a>) </li>
</ul>
<div>
<div style="text-align: justify;">
<b><u>Installing and booting up Minishift:</u></b></div>
</div>
<div>
<br /></div>
<div>
<div style="text-align: justify;">
Minishift installation instructions can be found at (<a href="https://docs.okd.io/latest/minishift/getting-started/installing.html">https://docs.okd.io/latest/minishift/getting-started/installing.html</a>). After installation just run to prepare the cluster:</div>
</div>
<div>
<br /></div>
<div>
<b><u>Creating the operator:</u></b></div>
<div>
<br />
<div style="text-align: justify;">
The first thing to do is prepare the layout for the <b>Operator</b>. Since we are going to create the <b>Operator</b> in Go, you need to create it in your <i>GOPATH</i> directory:</div>
<br />
<script src="https://gist.github.com/lordofthejars/e0a1c7f6386a65ec0f383d7d3c179dd2.js"></script>
<br />
<div style="text-align: justify;">
Then we need to add a new custom resource definition to this project, which will be responsible for defining how our custom resources look like:</div>
<br />
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/0440c90ca7af77eee7958d04135ac6a8.js"></script> <vr> We are generating a custom resource definition for a custom type called <i>Hello.</i><br />
</vr></div>
</div>
<div>
<br />
<div style="text-align: justify;">
Then we need to define the parameters that you want to set to the custom resource. These are the parts that you want to be different in every deployment. Let's say the version number or the message to print.</div>
<br />
<div style="text-align: justify;">
So let's define the spec and the status object.</div>
<br />
Open <i>pkg/apis/hello/v1alpha1/hello_types.go</i> and add next fields:<br />
<br />
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/2811f9433ef79b0a908b0d92e05837dc.js"></script> You define in <i>HelloSpec</i> struct a field called <i>Message</i> that will contain the message you want to be printed when the container is started.</div>
<br />
<div style="text-align: justify;">
Then you need to run next command to update the generated code: <i>operator-sdk generate k8s</i></div>
<i><br /></i>
<br />
<div style="text-align: justify;">
<i>Last part regarding code is to generate a controller which will be responsible for watching and reconciling our <i>Hello</i> resource. So run next command:</i></div>
<br />
<br />
<i>operator-sdk add controller --api-version=hello.lordofthejars.com/v1alpha1 --kind=Hello</i><br />
<br />
<div style="text-align: justify;">
The important file created after running this command is at <i>pkg/controller/hello/hello_controller.go </i>and in the next snippet you can see the important bits for our example:</div>
<br />
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/606c8c6b0c0253d062cdb18c176e5a72.js"></script> <i>Reconcile</i> method reads the state of the cluster for our <i>Hello</i> object and makes any changes based on the sate and what is in the spec object.</div>
<br />
<div style="text-align: justify;">
The next important piece is the method <i>newPodForCR</i> which is a custom method that generates programmatically the resource that we want to use. You can think about it as the template, where you define the schema, and you use the <i>Hello</i> kind to fill the empty spaces. Notice that in this method there is the <i>cr</i> variable which is used to get the values from the custom resource. Let's modify this method to adapt to our requirements.</div>
<br />
<script src="https://gist.github.com/lordofthejars/a9fc20a5190304b0c1a67da8f14bb540.js"></script>
<u><b>Installing the Operator:</b></u><br />
<br />
<div style="text-align: justify;">
Then you need to install the custom resource to the cluster, build the <b>Operator</b> Docker image and push it to Docker registry:</div>
<br />
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/5a7a4b43a5947ead1311a885471b487b.js"></script> After that, you need to update the operator's definition to use the created image. Open <i>deploy/operator.yaml and change REPLACE_IMAGE </i>tag to<i> lordofthejars/hello-operator:v0.0.1</i></div>
<i><br /></i>
<br />
<div style="text-align: justify;">
<i>And finally, we just need to create all <b>Operator</b> resources into the cluster:</i></div>
<br />
<br />
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/ce3bdec5fd7f808730fc7d18cf4e0252.js"></script> If you run<i> kubectl get pods</i> now, you'll see the <b>Operator</b> deployed in the cluster (<i>hello-operator-6d5559b65f-5zjg2 1/1 Running 0 25s</i>).</div>
<br />
<div style="text-align: justify;">
Now that we have everything in place, it is time to see it in action. Create next file:</div>
<br />
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/37afe0b1563b390b526e4190010d70e1.js"></script> This is our resource where we are only specifying the <i>message</i> to be printed.</div>
<br />
<div style="text-align: justify;">
And finally, run <i>oc/kubectl apply -f deploy/crds/hello_v1alpha1_hello_cr.yaml</i></div>
<br />
<div style="text-align: justify;">
Then you can check the log message by running <i>oc/kubectl logs example-hello-pod</i></div>
<i><br /></i>
<br />
<div style="text-align: justify;">
<i>To remove the resource, you just need to do as usually <i>oc/kubectl delete -f deploy/crds/hello_v1alpha1_hello_cr.yaml </i></i></div>
<br />
<br />
<div style="text-align: justify;">
Now just update the <i>hello_v1alpha1_hello_cr.yaml</i> file to another message and apply the resource again. See the logs and boom the new message is printed.</div>
<br />
<br />
<div style="text-align: justify;">
So notice that now we are not doing a copy-paste anymore, we just create a file with the configurable parts, and that's all, everything else is managed by the operator.</div>
<br />
<b><u>Conclusions:</u></b><br />
<br />
This is a really simple example, but you get the idea of how powerful is <b>Operators</b> and how they can simplify the way you deploy applications on <b>Kubernetes</b>.<br />
<br />
We keep learning,<br />
Alex<br />
<br />
<blockquote class="tr_bq">
I don't know why you're not fair, I give you my love, but you don't care, So what is right and what is wrong?, Gimme a sign (What is Love - Haddaway)</blockquote>
<br />
Music: <a href="https://www.youtube.com/watch?v=HEXWRTEbj1I">https://www.youtube.com/watch?v=HEXWRTEbj1I</a><br />
Follow me at<a href="https://twitter.com/alexsotob"> https://twitter.com/alexsotob</a><br />
<br />
<i><br /></i></div>
</div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com72tag:blogger.com,1999:blog-19517292.post-85837169258592296502019-01-15T15:21:00.000+01:002019-01-15T15:21:59.082+01:00SerenityBDD for clean Rest API tests<div class="separator" style="clear: both; text-align: center;">
<a href="https://avatars0.githubusercontent.com/u/9455201?s=200&v=4" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="200" data-original-width="200" height="200" src="https://avatars0.githubusercontent.com/u/9455201?s=200&v=4" width="200" /></a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: justify;">
<b><a href="https://serenity-bdd.github.io/theserenitybook/latest/index.html">Serenity BDD</a></b> helps you write cleaner and more maintainable automated acceptance and regression tests faster. As his name suggests, it is a tool for BDD, but in this post, I am going to show you that can be used standalone (no <i>Cucumber </i>or <i>JBehave</i> specs) just JUnit for testing Rest APIs.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<b>Serenity BDD</b> also implements the screenplay pattern, which encourages good testing habits and improves the readability of the tests.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Although <b>Serenity BDD</b> was created first with built-in support for web testing with <i>Selenium</i> 2, currently it also supports Rest API testing by using <i>Rest-Assured</i> out-of-the-box.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: justify;">
So let's see how to write some tests using <b>Serenity BDD</b> for Rest API services. For this example, we are going to use <a class="" href="https://reqres.in/">ReqRes</a> service which provides a simple Rest service with fake data, so you don't need to create a demo service anymore, just use one. We are going to create some integration test for this service which we can call it <i>User Service</i>.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
At the end of the post, you'll be able to get the source code of the example, but for now, let's see the important bits.</div>
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/8a5ec5514e49e62557d246de67c47254.js"></script>
</div>
<div style="text-align: justify;">
In <i>@Befor</i>e section, we are just setting the URL of the service under test. In this case, the URL is set by using a system property called <i>restapi.baseurl</i>, but of course there are other ways to do that like using <i>serenity.conf</i> or <i>serenity.properties</i> files.<br />
Then you also define the actor. This has the responsibility of doing the action, in this case, we named <i>Security Service</i> because in our example we suppose that <i>the Security Service</i> is the consumer and the <i>User Service</i> (implemented by <i>ReqRes</i>) is the provider.<br />
<br />
Then we are defining two tests, and notice how is the actor who is responsible for doing the actions. <i>attemptsTo</i> (for doing the request), and <i>should</i> (for asserting the response). As you can see every test now it is really readable on what is the purpose. Check for example the <i>find_an_individual_user()</i> test method. You can read almost as natural language something like "<i>security service attempts to find a user with id 1 and it should see that the response is (...)</i>".<br />
<br />
And if you are curious, the <i>FindAUser</i> class looks like;<br />
<br />
<script src="https://gist.github.com/lordofthejars/767a266c7494216dd2705d5b4baae979.js"></script>
You only take <i class="">performAs</i> method to implement the logic to do the request (in this case a <i>Get</i> method). Similar is done with other <i>Http</i> methods.<br />
<br />
So you can see that it is not mandatory to use BDD approach for using <b>Serenity BDD</b>, you can use it for integration tests, without having to use any Http client nor REST-Assured directly with the big win of creating readable tests.<br />
<br />
But there is one more thing to fell in love with <b>Serenity BDD</b>, and it is the reports it generates. If you start using it for BDD tests you'll see how powerful are its reports, being the live-documentation dream a reality, but if you are using just for integration tests, the generated reports are still impressive.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-EFIuvP3hOUQ/XCigwJcvUSI/AAAAAAAACXo/ZKuwTDhhXhYzlAMp8KOC7ATRhSxVJE4MQCLcBGAs/s1600/Screenshot%2B2018-12-30%2Bat%2B11.40.01.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="793" data-original-width="1600" height="158" src="https://4.bp.blogspot.com/-EFIuvP3hOUQ/XCigwJcvUSI/AAAAAAAACXo/ZKuwTDhhXhYzlAMp8KOC7ATRhSxVJE4MQCLcBGAs/s320/Screenshot%2B2018-12-30%2Bat%2B11.40.01.png" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-DXWKIerSlug/XCigwNKYHZI/AAAAAAAACXk/01TM3VS5El86Z-rwfBWpX_fT14N_G9FGgCLcBGAs/s1600/Screenshot%2B2018-12-30%2Bat%2B11.40.15.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="893" data-original-width="1600" height="178" src="https://3.bp.blogspot.com/-DXWKIerSlug/XCigwNKYHZI/AAAAAAAACXk/01TM3VS5El86Z-rwfBWpX_fT14N_G9FGgCLcBGAs/s320/Screenshot%2B2018-12-30%2Bat%2B11.40.15.png" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-vgCsVX7XGgA/XCigv23PpkI/AAAAAAAACXs/bBWdYPGBSgAVnNXXxpw0QFDXtLWLFaeHACLcBGAs/s1600/Screenshot%2B2018-12-30%2Bat%2B11.40.45.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="893" data-original-width="1600" height="178" src="https://3.bp.blogspot.com/-vgCsVX7XGgA/XCigv23PpkI/AAAAAAAACXs/bBWdYPGBSgAVnNXXxpw0QFDXtLWLFaeHACLcBGAs/s320/Screenshot%2B2018-12-30%2Bat%2B11.40.45.png" width="320" /></a></div>
<br />
So you can have a quick overview of how your integration tests for a given service is behaving.<br />
<br />
<b>Serenity BDD</b> is a really good choice for starting using BDD correctly, providing live-documentation to get the current state of the project and it integrates really well with <i>Cucumber/JBehave</i> as well as <i>Selenium</i> and <i>REST-Assured</i>. But if you are not into BDD or doing BDD-ish, then Serenity BDD is still a solution for just e2e tests (in case of monoliths) or integration tests.<br />
<br />
Source code: <a href="https://github.com/lordofthejars/ReqRes-Serenity">https://github.com/lordofthejars/ReqRes-Serenity</a> (<i>mvn clean verify -Pdemo</i>)<br />
<br />
We keep learning,<br />
Alex<br />
<blockquote class="tr_bq">
Per tu, no sóc un dels teus amants, però creuo l'Himàlaia per tu, i robaré dotze diamants, un per cada lluna plena (Fins que arribi l'alba - Els Catarres)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=Z5LVw2abUlw">https://www.youtube.com/watch?v=Z5LVw2abUlw</a><br />
Follow me at <a href="https://twitter.com/alexsotob">https://twitter.com/alexsotob</a><br />
<div>
<br /></div>
<br />
<br />
<br /></div>
<div style="text-align: justify;">
<br /></div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com14tag:blogger.com,1999:blog-19517292.post-37457213954023649342019-01-08T15:07:00.001+01:002019-01-11T08:21:00.094+01:00Auto-numbered Callouts in Asciidoctor<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-0zZFksiETU8/XDhD0i9L0VI/AAAAAAAACYQ/fhNgIwctGwMjm3roAmnJAmNhEcZS2mt9QCLcBGAs/s1600/logo-outline-color.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="375" data-original-width="375" height="200" src="https://3.bp.blogspot.com/-0zZFksiETU8/XDhD0i9L0VI/AAAAAAAACYQ/fhNgIwctGwMjm3roAmnJAmNhEcZS2mt9QCLcBGAs/s200/logo-outline-color.png" width="200" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div style="text-align: left;">
<div style="text-align: justify;">
<b>Asciidoctor 1.5.8</b> comes with a really nice feature which is called autonumber callouts, so you do not have to specify the number of the callout, but just a generic character (.) and then at rendering time, Asciidoctor will set the name correctly.</div>
</div>
<div style="text-align: left;">
<div style="text-align: justify;">
<br /></div>
</div>
<div style="text-align: left;">
<div style="text-align: justify;">
See the next example shows the number and auto-number feature for callouts:</div>
</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<script src="https://gist.github.com/lordofthejars/3b4961e5e83a5fa60eb3bb5c0e443183.js"></script>
</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
And the output looks like:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-rA5-N4Av1B0/XCYyvB8PV1I/AAAAAAAACXQ/JZzLR9zYJI0GdotcGzGxh4ZL-YeaEnFrACLcBGAs/s1600/Screenshot%2B2018-12-28%2Bat%2B15.17.44.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1246" data-original-width="1336" height="372" src="https://2.bp.blogspot.com/-rA5-N4Av1B0/XCYyvB8PV1I/AAAAAAAACXQ/JZzLR9zYJI0GdotcGzGxh4ZL-YeaEnFrACLcBGAs/s400/Screenshot%2B2018-12-28%2Bat%2B15.17.44.png" width="400" /></a></div>
<div style="text-align: justify;">
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
So as you can see the output is exactly the same in both cases, but in the first case the number of the callout is static meanwhile in the second one autonumber feature is used.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Using autonumbering feature is really useful when you've got big blocks of code where you might introduce some callouts into already defined callouts which means a shifting of all of them.</div>
<div style="text-align: justify;">
Meanwhile, with the first approach means you need to go to every callout and increase the number manually, in the later one you only need to add a new callout and that's all, you do not need to go manually increasing the callout number.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
We keep learning,</div>
<div style="text-align: justify;">
Alex.</div>
<div style="text-align: justify;">
<blockquote class="tr_bq">
My body is burning, it starts to shout, Desire is coming, it breaks out loud (Rock You Like Hurricane - Scorpions)</blockquote>
</div>
<div style="text-align: justify;">
Music: <a href="https://www.youtube.com/watch?v=AQRprgbYdTw">https://www.youtube.com/watch?v=AQRprgbYdTw</a><br />
Follow me at <a href="https://twitter.com/alexsotob">https://twitter.com/alexsotob</a></div>
<br /></div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com60tag:blogger.com,1999:blog-19517292.post-19603536600085698492018-11-19T17:51:00.000+01:002018-11-19T17:51:04.096+01:00Continuous Documentation with Antora and Travis<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-b9UNJTSUElc/W_LgOgIX1wI/AAAAAAAACUc/W9mSc-nT_bU325-48rcfK-G6VfqFnCCawCLcBGAs/s1600/antora-gitlab.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="240" data-original-width="240" height="200" src="https://1.bp.blogspot.com/-b9UNJTSUElc/W_LgOgIX1wI/AAAAAAAACUc/W9mSc-nT_bU325-48rcfK-G6VfqFnCCawCLcBGAs/s200/antora-gitlab.png" width="200" /></a></div>
<div style="text-align: justify;">
<div style="text-align: justify;">
<a href="https://antora.org/">Antora</a> is a documentation pipeline that enables docs, product, and engineering teams to create, manage, remix, and publish documentation sites composed in AsciiDoc and sourced from multiple versioned content repositories.</div>
</div>
<div style="text-align: justify;">
<div style="text-align: justify;">
<br /></div>
</div>
<div style="text-align: justify;">
<div style="text-align: justify;">
You can see several examples out there from <i>Couchbase</i> documentation to <i>Fedora</i> documentation. And of course, <b>Antora</b> documentation is used to generate Antora documentation. You can see it <a href="https://docs.antora.org/antora/1.1/">here</a>.</div>
</div>
<div style="text-align: justify;">
<div style="text-align: justify;">
<br /></div>
</div>
<div style="text-align: justify;">
<div style="text-align: justify;">
So basically we have our project with documents in <i>adoc</i> format. Then what we want is regenerating the documentation every time a PR is merged to master.</div>
</div>
<div style="text-align: justify;">
<div style="text-align: justify;">
<br /></div>
</div>
<div style="text-align: justify;">
<div style="text-align: justify;">
In our project, we are using <i>Travis-CI</i> as CI server, so I am going to show you how we have done.</div>
</div>
<div style="text-align: justify;">
<div style="text-align: justify;">
<br /></div>
</div>
<div style="text-align: justify;">
<div style="text-align: justify;">
First of all, you need to create a<i> .travis.yml</i> file on the root of your project.</div>
</div>
<div style="text-align: justify;">
<div style="text-align: justify;">
<br /></div>
</div>
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/a4daf546c8d754a46385860213dcb69e.js"></script>
</div>
<div style="text-align: justify;">
<div style="text-align: justify;">
<br /></div>
</div>
<div style="text-align: justify;">
<div style="text-align: justify;">
First, we define what we want to use. In this case <i>docker</i> and <i>git</i>.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Then in <i>before_install </i>section, we are detecting if we need to regenerate documentation or not.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Basically, we are going to generate documentation in two conditions:</div>
<br />
<ol>
<li style="text-align: justify;">If commit message contains the word <b><i>doc</i></b>, then docs should be regenerated.</li>
<li style="text-align: justify;">If you have modified an <b><i class="">adoc</i></b> file from the <i><u>documentation</u></i> folder (or <i><u>README.adoc</u></i>) and the branch is <b><i>master</i></b>, then the docs should be regenerated.</li>
</ol>
<div style="text-align: justify;">
If any of these conditions are met, then we configure <i>git client</i> with <i class="">user</i>, <i>email</i> and <i>token</i> to be used for pushing the generated documentation. Notice that this information comes from environment variable defined in Travis console. Also, it is important to note that the documentation should be generated in <i class="">gh-pages</i> branch (since we are releasing to GitHub pages). For this reason, we are using <i>git worktree </i>which checkouts the <i>gh-pages</i> branch in <i class="">gh-pages</i> directory.</div>
<div>
<br /></div>
<div>
Then in <i>script</i> section, we are just using <b>Antora</b> docker image to render documentation.</div>
<div>
<br /></div>
<div>
Finally, we just need to enter into<i> gh-pages</i> directory, create a <i>.nojekyll</i> file to avoid Git Hub Pages thinks that this is a Jekyll site, and finally push the changes.</div>
<div>
<br /></div>
<div>
And then for PR merged, the documentation is automatically regenerated and published.</div>
<div>
<br /></div>
<div>
<b>Important</b>: This script is based on one done previously by Bartosz Majsak (@majson) for Asciidoctor. My task has been only adapting it to use Antora.</div>
<div>
<br /></div>
<div>
We keep learning,</div>
<div>
Alex.</div>
<br />
<blockquote class="tr_bq">
Y no me importa nada nada (nada), Que rías o que sueñes, que digas o que hagas, Y no me importa nada, Por mucho que me empeñe, estoy jugando y no me importa nada (No me importa nada - Luz Casal)</blockquote>
<br />
<div>
Music: <a href="https://www.youtube.com/watch?v=YkS-ttHspLs">https://www.youtube.com/watch?v=YkS-ttHspLs</a></div>
<div>
Follow me at <a href="https://twitter.com/alexsotob">https://twitter.com/alexsotob</a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<br />
<br />
<br /></div>
<div style="text-align: justify;">
<br /></div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com7tag:blogger.com,1999:blog-19517292.post-70683284909137883572018-10-03T15:09:00.000+02:002018-10-03T15:09:21.381+02:00Arquillian Chameleon Cheat Sheet<div class="separator" style="clear: both; text-align: center;">
<a href="http://ichef.bbci.co.uk/wwfeatures/wm/live/1280_640/images/live/p0/2z/fh/p02zfh0x.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://ichef.bbci.co.uk/wwfeatures/wm/live/1280_640/images/live/p0/2z/fh/p02zfh0x.jpg" data-original-height="400" data-original-width="800" height="160" width="320" /></a></div>
<br />
<div style="text-align: justify;">
<b><a href="https://github.com/arquillian/arquillian-container-chameleon">Arquillian Chameleon</a></b> simplifies how we can write container tests in Arquillian, it has been out there for several times, but now in this post, I share with you a refcard so you can print and take a quick overview of its functionalities.</div>
<div style="text-align: justify;">
<br /></div>
<br />
<div style="display: block; font-family: "helvetica" , "arial" , sans-serif; font-size: 14px; font-stretch: normal; font-style: normal; font-variant: normal; font-weight: normal; line-height: normal; margin: 12px auto 6px auto;">
<a href="https://www.scribd.com/document/390019818/Arquillian-Chameleon-Cheat-Sheet#from_embed" style="text-decoration: underline;" title="View Arquillian Chameleon Cheat Sheet on Scribd">Arquillian Chameleon Cheat ...</a> by <a href="https://www.blogger.com/undefined#from_embed" style="text-decoration: underline;" title="View 's profile on Scribd"></a> on Scribd</div>
<br />
<br />
Special thanks to <a href="https://twitter.com/Mogztter">https://twitter.com/Mogztter</a> for make it possible with its contribution to asciidoctor.js.<br />
<br />
We keep learning,<br />
Alex<br />
<blockquote class="tr_bq">
You don't have to believe no more, Only got four hours, To learn your manners, Never felt so close to you before (King George - Dover)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=wbM9RtOGdKE">https://www.youtube.com/watch?v=wbM9RtOGdKE</a><br />
Follow me at <a href="https://twitter.com/alexsotob">https://twitter.com/alexsotob</a><br />
<br />Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com44tag:blogger.com,1999:blog-19517292.post-11816923591613592102018-08-13T14:01:00.000+02:002018-08-13T14:01:21.994+02:00Java Iterator to Java 8 Stream<div class="separator" style="clear: both; text-align: center;">
<a href="https://images.pexels.com/photos/582590/pexels-photo-582590.jpeg?auto=compress&cs=tinysrgb&h=350" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="350" data-original-width="525" height="133" src="https://images.pexels.com/photos/582590/pexels-photo-582590.jpeg?auto=compress&cs=tinysrgb&h=350" width="200" /></a></div>
<br />
<div style="text-align: justify;">
Sometimes during my work, I need to integrate with other libraries which they return an <b>Iterator</b> object instead of a list. This is fine from the point of view of libraries but it might be a problem when you want to use <b>Java 8 streams</b> on the returned iterator. There is one way to transform the <b>Iterator</b> to <b>Iterable</b> and then easily to stream.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Since all the time I need to remember how to do it, I decided to share the snippet here.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/429ec7967462ac1282ad8f348f1de7fb.js"></script>
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
In the example, first of all, we have an <b>Iterator</b> class. Since <b>Iterator</b> cannot be used as a stream but an <b>Iterable</b> can do, we just create a new <b>Iterable</b> class which overrides its <i>iterator()</i> method to return the <b>Iterator</b> we want to stream.<br />
<br />
Then we have an <b>Iterable</b> which is not streamable yet. So what we need to do is to use <b>StreamSupport</b> class to convert the Iterable to a <b class="">Stream</b>.<br />
<br />
And that's all then you can use all streaming operations without any problem.<br />
<br />
We keep learning,<br />
Alex.<br />
<blockquote class="tr_bq">
Prefereixo que em passis la birra que em tiris la canya, Perdona'm si em ric però es que em fas molta gràcia, Lligar no es lo teu, Em sap molt de greu (Lligar no és lo teu - Suu)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=fWNqMjAVNto">https://www.youtube.com/watch?v=fWNqMjAVNto</a><br />
Follow me at <a href="https://twitter.com/alexsotob">https://twitter.com/alexsotob</a></div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com28tag:blogger.com,1999:blog-19517292.post-78573681653088370672018-06-07T18:29:00.000+02:002018-06-07T18:29:13.654+02:00Spring Boot + Cockroach DB in Kubernetes/OpenShift<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-DvI0i_YgvtY/WxT8sljW-1I/AAAAAAAACNQ/O9PFQhwkPGY5ChFlWzEZa96V56PdhFtwQCLcBGAs/s1600/spring-boot-logo.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="315" data-original-width="600" height="105" src="https://2.bp.blogspot.com/-DvI0i_YgvtY/WxT8sljW-1I/AAAAAAAACNQ/O9PFQhwkPGY5ChFlWzEZa96V56PdhFtwQCLcBGAs/s200/spring-boot-logo.png" width="200" /></a></div>
<br />
<div style="text-align: justify;">
In my previous <a href="http://www.lordofthejars.com/2018/05/cockroachdb-cloud-native-sql-database.html">post</a>, I showed why <b>CockroachDB</b> might help you if you need a cloud native SQL database for your application. I explained how to install it in <b>Kubernetes/OpenShift</b> and how to validate that the data is replicated correctly.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
In this post, I am going to show you how to use <b>Cockroach DB</b> in a <b>Spring Boot</b> application. Notice that <b>Cockroach DB</b> is compatible with PostgresSQL driver, so in terms of configuration is almost the same.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
In this post, I assume that you have already a <b>Cockroach DB</b> cluster running in <b>Kubernetes</b> cluster as explained in my previous post.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
For this example, I am using Fabric8 Maven Plugin to smoothly deploy a <b>Spring Boot</b> application to <b>Kubernetes</b> without having to worry so much about creating resources, creating Dockerfile and so on. Everything is automatically created and managed.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
For this reason, <i>pom.xml</i> looks like:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/8b5ad635976b8f6036860e214d155b0e.js"></script>
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Notice that apart from defining Fabric8 Maven Plugin I am also defining to use Spring Data JPA to make the integration between <b>Spring Boot </b>and JPA easier from the point of view of the developer.</div>
<div style="text-align: justify;">
<br /></div>
Then you need to create a JPA entity and Spring Data Crud repository to interact with JPA.<br />
<br />
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/1fc9a5ad27164ff7b4c348d754de4bd3.js"></script> Also, we need to create a controller who is responsible to get incoming requests, use the repository to make queries to DB and return results back to the caller.</div>
<br />
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/d935973b6edea12f76d40bf79e4986f2.js"></script> Finally, you need to configure JPA to use the desired driver and dialect. In case of <b>Spring Boot</b> this is done in <i>application.properties</i> file.</div>
<br />
<script src="https://gist.github.com/lordofthejars/681bd851d96841005286282c75e31b8a.js"></script>
<br />
<div style="text-align: justify;">
The most important part here is that we need to use the PostgeSQL94 dialect. Notice that in <i>url</i>, we are using the postgresql jdbc url form. That's fine, since <b>Cockroach</b> uses the Postgres driver.</div>
<br />
<div style="text-align: justify;">
Now we need to create the database (<i>customers</i>) and the user (<i>myuser</i>) as configured in application.properties. To make it so, you just need to run cockroach shell and run some SQL commands:</div>
<br />
<script src="https://gist.github.com/lordofthejars/a2f336c8503050baa794f04936e3c326.js"></script>
<br />
<div style="text-align: justify;">
Finally, you can deploy the application by running <i>mvn clean fabric8:deploy</i>. After that, the first time might take longer since needs to pull Docker images, you can start sending queries to the service.</div>
<br />
<div style="text-align: justify;">
As you can see it is really easy to start using a cloud-native DB like <b>Cockroach DB</b> in <b>Spring Boot</b>. If you want you can do exactly the same as in my previous <a class="" href="http://www.lordofthejars.com/2018/05/cockroachdb-cloud-native-sql-database.html">post</a> and start running queries to each of the nodes to validate that data is available correctly.</div>
<br />
Code: <a href="https://github.com/lordofthejars/springboot-cockroach">https://github.com/lordofthejars/springboot-cockroach</a><br />
<br />
We keep learning,<br />
Alex.<br />
<blockquote class="tr_bq">
Dôme épais, le jasmin, à la rose s'assemble, rive en fleurs, frais matin, nous appellent ensemble. (Flower Duet - Lakmé - Leo Delibes)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=Vf42IP__ipw">https://www.youtube.com/watch?v=Vf42IP__ipw</a><br />
Follow me at <a href="https://twitter.com/alexsotob">https://twitter.com/alexsotob</a><br />
<br />
<br />
<br />Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com92tag:blogger.com,1999:blog-19517292.post-80338889503079807482018-05-29T17:46:00.002+02:002018-05-29T17:46:46.348+02:00CockroachDB. A cloud native SQL database in Kubernetes.<div class="separator" style="clear: both; text-align: center;">
<a href="https://www.cockroachlabs.com/uploads/2016/10/running-cockroachdb-on-kubernetes.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="236" data-original-width="529" height="142" src="https://www.cockroachlabs.com/uploads/2016/10/running-cockroachdb-on-kubernetes.png" width="320" /></a></div>
<br />
<b><br /></b>
<div style="text-align: justify;">
<b><a href="https://www.cockroachlabs.com/">CockroachDB 2.0</a></b> has just been released. For those who don't know what it is, it can be summarized as a SQL database for the cloud era. One of the best things about <b>CockroachDB</b> is that it automatically scales, rebalances and repairs itself without sacrificing the SQL language. Moreover, <b>Cockroach</b> implements ACID transactions, so your data is always in a known state.</div>
<br />
<div style="text-align: justify;">
In this post, I am going to explain how to install it in Kubernetes/OpenShift, insert some data and validate that it has been replicated in all nodes. In next post, I am going to show you how to use it with Spring Boot + JPA.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The first thing you need to have is a Kubernetes/OpenShift cluster to be used. You can use Minikube or Minishift for this purpose. In my case, I am using Minishift but I will provide equivalent commands for Minikube.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
After having everything installed, you need to launch the <b>Cockroach</b> cluster.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: left;">
In case of <b>Kubernetes</b>: <i>kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml</i></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: left;">
In case of <b>OpenShift</b>: <i>oc apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml</i></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Then you need to initialize the cluster:</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
In case of <b>Kubernetes</b>: <i>kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml</i></div>
<div style="text-align: left;">
<i><br /></i></div>
<div style="text-align: left;">
In case of <b>OpenShift</b><i>: oc apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml</i></div>
<div style="text-align: left;">
<i><br /></i></div>
<div style="text-align: left;">
Then let's configure the cluster so we can access the admin UI:</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
In case of <b>Kubernetes</b>: <i>kubectl port-forward cockroachdb-0 8080</i></div>
<div style="text-align: left;">
<i><br /></i></div>
<div style="text-align: left;">
In case of <b>OpenShift</b><i>: oc expose svc cockroachdb-public --port=8080 --name=r1</i></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: justify;">
Now let's create a database and a table and see how it is replicated across all the cluster easily. <b>Cockroach</b> comes with a service that offers a load-balanced virtual IP for clients to access the database.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
In case of <b>Kubernetes</b>: <i>kubectl run cockroachdb -it --image=cockroachdb/cockroach --rm --restart=Never -- sql --insecure --host=cockroachdb-public</i></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
In case of <b>OpenShift</b>: <i>oc run cockroachdb -it --image=cockroachdb/cockroach --rm --restart=Never -- sql --insecure --host=cockroachdb-public</i></div>
<div style="text-align: left;">
<i><br /></i></div>
<div style="text-align: left;">
And finally in the opened console just type some <i>SQL</i> calls:</div>
<div style="text-align: left;">
<i><br /></i></div>
<div style="text-align: left;">
<i>create database games;</i></div>
<div style="text-align: left;">
<i>use games;</i></div>
<div style="text-align: left;">
<i>create table game (id int, title varchar(30));</i></div>
<div style="text-align: left;">
<i>insert into game values (1, 'The Secret of Monkey Island');</i></div>
<div style="text-align: left;">
<i><br /></i></div>
<div style="text-align: left;">
So far, we have a new database, table, and entry in <b>CockroachDB</b>. Open admin UI, push <i>Databases</i> and you'll see something like this:</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-sRWkNVMW-Y8/Ww021EuWDQI/AAAAAAAACMI/6HcrrgzSeD4NR89VyS-JrnzOboosoc2kgCLcBGAs/s1600/tables.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="407" data-original-width="1468" height="176" src="https://1.bp.blogspot.com/-sRWkNVMW-Y8/Ww021EuWDQI/AAAAAAAACMI/6HcrrgzSeD4NR89VyS-JrnzOboosoc2kgCLcBGAs/s640/tables.png" width="640" /></a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: justify;">
<i><br /></i></div>
<div style="text-align: justify;">
You can see that the database and the table have been created. Now let's see how we can know that everything has been replicated correctly. Push Overview and you'll see something like:</div>
<div style="text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-7zxtNBagfZQ/Ww1E6NtEpeI/AAAAAAAACMU/E3OauKfJtZo72nhnCbhJCx_xWv41kMccgCLcBGAs/s1600/livenodes.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="318" data-original-width="1376" height="147" src="https://3.bp.blogspot.com/-7zxtNBagfZQ/Ww1E6NtEpeI/AAAAAAAACMU/E3OauKfJtZo72nhnCbhJCx_xWv41kMccgCLcBGAs/s640/livenodes.png" width="640" /></a></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
So pay attention to <i>Replicas</i> column. In all nodes, the number is exactly the same number, this means that all data in the cluster has been replicated X times.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Now let's increase by one the number of replicas and just refresh the page to see that the new node initially has not the same replica count.</div>
<div>
<br /></div>
<div>
In case of <b>Kubernetes</b>: <i>kubectl scale statefulset cockroachdb --replicas=4</i></div>
<div>
<i><br /></i></div>
<div>
In case of <b>OpenShift</b><i>: oc scale statefulset cockroachdb --replicas=4</i></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-RL93xr9y3wQ/Ww1HUnkmo6I/AAAAAAAACMg/3S5V8N0O744Iz3rRKVMI2LS8-VVwvLWUwCLcBGAs/s1600/Screen%2BShot%2B2018-05-29%2Bat%2B14.27.15.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="333" data-original-width="1438" height="148" src="https://2.bp.blogspot.com/-RL93xr9y3wQ/Ww1HUnkmo6I/AAAAAAAACMg/3S5V8N0O744Iz3rRKVMI2LS8-VVwvLWUwCLcBGAs/s640/Screen%2BShot%2B2018-05-29%2Bat%2B14.27.15.png" width="640" /></a></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Another thing you can do is to just enter each container and validate that when connecting to localhost, the inserted data is there.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
In case of <b>Kubernetes</b>: <i>kubectl exec -it cockroachdb-0 /bin/bash</i></div>
<div style="text-align: left;">
<i><br /></i></div>
<div style="text-align: left;">
In case of <b>OpenShift</b>: <i>oc exec -it cockroachdb-0 /bin/bash</i></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: justify;">
Then inside the container just run: <i>./cockroach dump games --insecure</i></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
And you will see that CLI connects by default to the current node (localhost) and dumps the content of <i>games</i> db.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Repeat the same with other nodes <i>cockroachdb-1 </i>and<i> </i><i>cockroachdb-2</i> and you should see exactly the same.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
So as you can see, it is really easy to use SQL in scale way thanks to <b>Cockroach DB</b>. In next post, we are going to see how to integrate Spring Boot + JPA with<b> Cockroach DB</b>, and deploying it into Kubernetes. </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
We keep learning,</div>
<div style="text-align: left;">
Alex</div>
<blockquote class="tr_bq">
I can see a rainbow, In your tears as they fall on down, I can see your soul grow, Through the pain as they hit the ground (Rainbow - Sia)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=paXOkGMyG8M">https://www.youtube.com/watch?v=paXOkGMyG8M</a><br />
<br />
Follow me at <a href="https://twitter.com/alexsotob">https://twitter.com/alexsotob</a><br />
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<br /></div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com43tag:blogger.com,1999:blog-19517292.post-59784669912425415642018-03-26T12:49:00.001+02:002018-03-26T12:49:31.621+02:00Arquillian Chameleon. Simplifying your Arquillian tests.<div class="separator" style="clear: both; text-align: center;">
<a href="https://www.cbreptile.com/wp-content/uploads/2017/08/ambilobe-panther-chameleon-breeder.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="469" data-original-width="800" height="187" src="https://www.cbreptile.com/wp-content/uploads/2017/08/ambilobe-panther-chameleon-breeder.jpg" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<b>Arquillian Chameleon</b> was born to simplify the configuration of <b>Arquillian</b> tests. I am proud to announce that with version <b>1.0.0.CR2</b> we have not only simplified how to configure <b>Arquillian</b> tests but also how to write them.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
With this new release, three new simplifications have been added:</div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<ul>
<li style="text-align: justify;">You only need to use 1 (or at most 2 dependencies just in case you want to use auto-deployment feature)</li>
<li style="text-align: justify;">It is not necessary to add any dependency to define which application server you want to use to run tests. Even not necessary to use <i>arquillian.xml</i> file to define it.</li>
<li style="text-align: justify;">It is not necessary to use ShrinkWrap to build your package. You can still use it, but you can delegate the process of creating the package to a custom SPI.</li>
</ul>
<div style="text-align: justify;">
So let's start.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>Dependency</b></div>
<div class="separator" style="clear: both; text-align: left;">
<b><br /></b></div>
<div class="separator" style="clear: both; text-align: justify;">
You only need to add one dependency you don't need to add Arquillian dependency + container dependency anymore.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<script src="https://gist.github.com/lordofthejars/5c92b4121ee1b57eecbd75f26afbb84f.js"></script>
</div>
<div class="separator" style="clear: both; text-align: left;">
<b>Definition of container</b> </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Now to define a container you just need to use a special runner and special annotation:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<script src="https://gist.github.com/lordofthejars/05ed0da7eb9241a7db82000cd557d0b3.js"></script>
</div>
<div class="separator" style="clear: both; text-align: justify;">
You just need to use <b>ArquillianChameleon</b> runner and the special annotation <b>@ChameleonTarget</b> to define which container you want to use. In this example, <b>Wildfly 11</b> with the managed mode is configured.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
When running this test, classpath is going to be configured with <b>Arquillian Wildfly</b> dependency, download the application server and behave as any other <b>Arquillian</b> test.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
You can learn more about this feature at <a href="https://github.com/arquillian/arquillian-container-chameleon#arquillian-chameleon-runner">https://github.com/arquillian/arquillian-container-chameleon#arquillian-chameleon-runner</a></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<b>AutoDeployment</b></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<b>Arquillan</b> allows you to define a Java SPI to describe how the archive should be created. This effectively means that no <b>@Deployment</b> method is required if you provide an implementation which automatically creates the deployment file.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<b>Arquillian Chameleon</b> provides at this time two implementations:</div>
<div class="separator" style="clear: both; text-align: justify;">
</div>
<ol>
<li><i>File</i> which deploys an already created file. You need to set the location of the file.</li>
<li><i>Maven</i> which runs using embedded Maven the build of the project and the generated archive is used as deployment archive.</li>
</ol>
<div>
For this example, I am going to use a multi-module project as an example, but notice that if you create a none multimodule project, then defaults works perfectly.</div>
<div>
<br /></div>
<div>
<script src="https://gist.github.com/lordofthejars/6dba14dc2be6adf76ee2aadbc06678af.js"></script>
</div>
<br />
<div class="separator" style="clear: both; text-align: justify;">
Notice that depending on the method you choose (<i>File</i> or <i>Maven</i>) you need to add the implementation on classpath.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
In this case, I choose to use the Maven approach which means that the archive is generated by building all project.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Two things that are specific to this test and needs to be customized (instead of defaults) because of the example.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
First one is the pom location. By default, the <i>@MavenBuild</i> annotation uses the pom.xml where the test is executed. In case of multimodule project, you don't want to run the build from module where test is defined but from the root of the project, so you get a complete archive with all dependencies. For this case you need to set it where is located.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
The second one is where is the archive generated to be used to deploy. By default, you don't need to specify anything since in case of none multimodule project you are only generating one file. But in case of multimodule projects, you are generating multiple archives, so you need to specify which module contains the final archive.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
And that's all, when you run this test, <b>Arquillian</b> will download Wildfly, start it, runs the build to get the final deployment file (such as .war), deploy it and finally run the test.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Notice that also there is <i>@DeploymentParameters</i> annotation which is not mandatory to be used, but allows you to configure the deployment as you do with @Deployment annotation, such as setting a deployment name or changing the mode from a container (the default one) to as client.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
You can see full example at: <a href="https://github.com/lordofthejars/games-monolith/blob/master/gamepage/impl/src/test/java/org/lordofthejars/games/game/GameResourceRestApiTest.java">https://github.com/lordofthejars/games-monolith/blob/master/gamepage/impl/src/test/java/org/lordofthejars/games/game/GameResourceRestApiTest.java</a></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Also you can read more about autodeployment feature at <a href="https://github.com/arquillian/arquillian-core/blob/master/docs/deployment-archives.adoc#deployment-archives-using-java-spi">https://github.com/arquillian/arquillian-core/blob/master/docs/deployment-archives.adoc#deployment-archives-using-java-spi</a></div>
<div class="separator" style="clear: both; text-align: justify;">
<b><br /></b></div>
<div class="separator" style="clear: both; text-align: justify;">
<b>Conclusions</b></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
You can see that everything has been simplified a lot. The idea is to offer a similar experience that you get when running a Spring tests.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
We keep learning,</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Alex</div>
<blockquote class="tr_bq" style="clear: both; text-align: justify;">
Not knowing what it was, I will not give you up this time, But darling, just kiss me slow, your heart is all I own, And in your eyes you're holding mine (Perfect - Ed Sheraan)</blockquote>
<div class="separator" style="clear: both; text-align: justify;">
Music: <a href="https://www.youtube.com/watch?v=2Vv-BfVoq4g">https://www.youtube.com/watch?v=2Vv-BfVoq4g</a></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Follow me at <a href="https://twitter.com/alexsotob">https://twitter.com/alexsotob</a></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<br />Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com17tag:blogger.com,1999:blog-19517292.post-65138163603227239402018-02-12T17:27:00.001+01:002018-02-12T17:27:20.644+01:00Repeatable Annotations in Java 8<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-aMKMfdupYlw/WnV4B4aNKAI/AAAAAAAAB_0/zoNsZudjj6M-xkz_z2-dysjl6er7QhaTwCLcBGAs/s1600/rinserepeat.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="319" data-original-width="666" height="95" src="https://2.bp.blogspot.com/-aMKMfdupYlw/WnV4B4aNKAI/AAAAAAAAB_0/zoNsZudjj6M-xkz_z2-dysjl6er7QhaTwCLcBGAs/s200/rinserepeat.png" width="200" /></a></div>
<br />
<div style="text-align: justify;">
With Java 8 you are able to repeat the same annotation to a declaration or type. For example, to register that one class should only be accessible at runtime by specific roles, you could write something like:</div>
<br />
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/17f3f2905b0a5d954e11b4ddcda36959.js"></script> Notice that now <i>@Role</i> is repeated several times. For compatibility reasons, repeating annotations are stored in a container annotation, so instead of writing just one annotation you need to write two, so in the previous case, you need to create <i>@Role</i> and <i>@Roles</i> annotations.</div>
<br />
<div style="text-align: justify;">
Notice that you need to create two annotations, one which is the "plural" part of the annotation where you set return type of <i>value </i>method to be an array of the annotation that can be used multiple times. The other annotation can be used multiple time in the scope where it is defined and must be annotated with @<i>Repeatable</i> annotation.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
This is how I did all the time since Java 8 allows to do it. But last week, during a code review my mate George Gastaldi pointed me out how they are implementing these repeatable annotations in <i>javax.validation</i> spec. Of course, it is not completely different but I think that looks pretty much clear from point of view implementation since everything is implemented within the same archive and also, in my opinion, the name looks much natural. </div>
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/509e67ec93720589e7c005a6c01f6c17.js"></script>
</div>
<div style="text-align: justify;">
<br />
Notice that now everything is placed in the same archive. Since usually you only need to refer to <i>@Role </i>class, and not <i>@Roles</i> (now <i>@Role.List</i>) annotation you can hide this annotation as an inner annotation. Also in case of defining several annotations, this approach makes everything look more compact, instead of having of populating the hierarchy with "duplicated" classes serving the same purpose, you only create one.<br />
<br />
Of course, I am not saying that the approach of having two classes is wrong, at the end is about preferences since both are really similar. But after implementing repeatable annotations in this way, I think that it is cleaner and compact solution having everything defined in one class.<br />
<br />
We keep learning,<br />
Alex.<br />
<blockquote class="tr_bq">
Jo sóc l'hipopòtam, i crec que el lleó, ha de refrescar-se per estar molt millor (El Lleó Vergonyós - El Pot Petit)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=lYriMzzMsUw">https://www.youtube.com/watch?v=lYriMzzMsUw</a><br />
<br />
Follow me at <a href="https://twitter.com/alexsotob">https://twitter.com/alexsotob</a><br />
<br />
<br /></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<br /></div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com8tag:blogger.com,1999:blog-19517292.post-2398127236545904112018-01-08T15:26:00.001+01:002018-01-08T15:26:53.523+01:00Secret Rotation for JWT tokens<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.cardgamedb.com/forums/uploads/blog-0281295001389910674.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://www.cardgamedb.com/forums/uploads/blog-0281295001389910674.jpg" data-original-height="791" data-original-width="800" height="197" width="200" /></a></div>
<div style="text-align: center;">
<br /></div>
<br />
<div style="text-align: justify;">
When you are using <b>JSON Web Token</b> (<b>JWT</b>), or any other token technology that requires to sign or encrypt payload information, it is important to set an expiration date to the token, so if the token expires, you can either assume that this might be considered a security breach and you refuse any communication using this token, or you decide to enable the token by updating it with new expiry date.</div>
<br />
<div style="text-align: justify;">
But it is also important to use some kind of secret rotation algorithm, so the secret used to sign or encrypt a token is periodically updated, so if the secret is compromised the tokens leaked by this key is less. Also in this way you are decreasing the probability of a secret being broken.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
There are several strategies for implementing this, but in this post, I am going to explain how I implemented secret rotation in one project I developed some years ago to sign <b>JWT</b> tokens with HMAC algorithm.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I am going to show how to create a <b>JWT</b> token in Java.</div>
<br />
<script src="https://gist.github.com/lordofthejars/64f531a64634110cdfac20c756602269.js"></script>
<br />
<div style="text-align: justify;">
Notice that what you need to do here is creating an algorithm object setting HMAC algorithm and set a secret that is used to sign and verify instance.</div>
<br />
<div style="text-align: justify;">
So what we need is to rotate this algorithm instance every X minutes, so the probability of breaking the secret, and that the broken secret is still valid, becomes very low.</div>
<br />
<div style="text-align: justify;">
So how to rotate secrets? Well, with a really simple algorithm that everyone (even if you are not a crypto expert) can understand. Just using time.</div>
<br />
<div style="text-align: justify;">
So to generate the secret, you need a string, in the previous example was <i>secret </i>String, of course, this is not so secure, so the idea is to compose this secret string by a root (something we called the big bang part) + a shifted part time. In summary the secret is <i><bigbang></i>+<i><timeInMilliseconds></i></div>
<br />
Bing bang part has no mystery, it is just a static part, for example, <i>my_super_secret</i>.<br />
<br />
<div style="text-align: justify;">
The interesting part is the time part. Suppose you want to renew secret every second. You only need to do this:</div>
<br />
<script src="https://gist.github.com/lordofthejars/1d011017d839b787e4b23891bae89fab.js"></script> I am just putting 0s to milliseconds part, so if I run this I get something like:<br />
<br />
<i>1515091335543</i><br />
<i>1515091335500</i><br />
<i>1515091335500</i><br />
<br />
<div style="text-align: justify;">
Notice that although between second and third print, it has passed 50 milliseconds, the time part is exactly the same. And it will be the same during the same second.</div>
<br />
<div style="text-align: justify;">
Of course, this is an extreme example where the secret is changed every second but the idea is that you remove the part of the time that you want to ignore, and fill it with 0s. For this reason, first, you are dividing the time and then multiply by the same number.</div>
<br />
<div style="text-align: justify;">
For example, suppose that you want to rotate the secret every 10 minutes you just need to divide and multiply for 600000.</div>
<br />
<div style="text-align: justify;">
There are two problems with this approach that can be fixed although one of them is not really a big issue.</div>
<br />
<div style="text-align: justify;">
The first one is that since you are truncating the time if you want to change the secret every minute and for example, the first calculation occurs in the middle of a minute, then for just this initial case, the rotation will occur after 30 seconds and not 1 minute. Not a big problem and in our project, we did nothing to fix it.</div>
<br />
<div style="text-align: justify;">
The second one is what's happening with tokens that were signed just before the secret rotation, they are still valid and you need to be able to verify them too, not with the new secret but with previous one.</div>
<br />
<div style="text-align: justify;">
To fix this, what we did was to create a valid window, where the previous valid secret was also maintained. So when the system receives a token, it is verified with the current secret, if it passes then we can do any other checks and work with it, if not then the token is verified by the previous secret. If it passes, the token is recreated and signed with the new secret, and if not then obviously this token is invalid and must be refused.</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-v4X4hsmMwlk/WlJaeqQc3-I/AAAAAAAAB8w/ZGw-MuJmr5kLRpSb8EZpTDjnjaGu8SeCACLcBGAs/s1600/Verifying%2BJWT%2Btoken.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="642" data-original-width="372" height="400" src="https://3.bp.blogspot.com/-v4X4hsmMwlk/WlJaeqQc3-I/AAAAAAAAB8w/ZGw-MuJmr5kLRpSb8EZpTDjnjaGu8SeCACLcBGAs/s400/Verifying%2BJWT%2Btoken.png" width="231" /></a></div>
<br />
<br />
<div style="text-align: justify;">
To create the algorithm object for <b>JWT</b> you only need to do something like:</div>
<div style="text-align: justify;">
<br /></div>
<script src="https://gist.github.com/lordofthejars/8c2c8d43b761e810bd4a8c261a5017f4.js"></script>
<br />
<div style="text-align: justify;">
What I really like about this solution is:</div>
<br />
<ul>
<li style="text-align: justify;">It is clean, no need for extra elements on your system.</li>
<li style="text-align: justify;">No need for triggered threads that are run asynchronously to update the secret.</li>
<li style="text-align: justify;">It is really performant, you don't need to access an external system.</li>
<li style="text-align: justify;">Testing the service is really easy.</li>
<li style="text-align: justify;">The process of verifying is responsible for rotating the secret.</li>
<li style="text-align: justify;">It is really easy to scale, in fact, you don't need to do anything, you can add more and more instances of the same service and all of them will rotate the secret at the same time, and all of them will use the same secret, so the rotating process is really stateless, you can scale up or down your instances and all instances will continue to be able to verify tokens signed by other instances.</li>
</ul>
<br />
But of course there are some drawbacks:<br />
<br />
<ul>
<li style="text-align: justify;">You still need to share a part of the secret (the big bang part) to each of the services in a secure way. Maybe using Kubernetes secrets, Vault from Hashicorp or if you are not using microservices, you can just copy a file into a concrete location and when the service is up and running, read the big bang part, and then just remove it.</li>
<li style="text-align: justify;">If your physical servers are in different time zones, then using this approach might be more problematic. Also, you need that the servers are more or less synchronized. Since you are storing the previous token and current token, it is not necessary that they are synced in the same second and some seconds delay is still possible without any issue.</li>
</ul>
<br />
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
So we have seen a really simple way of rotating secrets so you can keep your tokens safer. Of course, there are other ways of doing the same. In this post, I just have explained how I did it in a monolith application we developed three years ago, and it worked really well.</div>
<br />
We keep learning,<br />
Alex.<br />
<blockquote class="tr_bq">
You just want attention, you don't want my heart, Maybe you just hate the thought of me with someone new, Yeah, you just want attention, I knew from the start, You're just making sure I'm never gettin' over you (Attention - Charlie Puth)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=nfs8NYg7yQM">https://www.youtube.com/watch?v=nfs8NYg7yQM</a><br />
<br />
Follow me at <a href="https://twitter.com/alexsotob">https://twitter.com/alexsotob</a>Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com12tag:blogger.com,1999:blog-19517292.post-24470496281845734102018-01-03T13:23:00.001+01:002018-01-08T15:42:39.773+01:00Cloud Native Applications with JWT<div class="separator" style="clear: both; text-align: center;">
<a href="https://cdn.vox-cdn.com/thumbor/C1SdoDActSv8tPONx_OjwEobUjw=/0x0:1004x753/1200x800/filters:focal(0x0:1004x753)/cdn.vox-cdn.com/uploads/chorus_image/image/49523369/20150428-cloud-computing.0.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="533" data-original-width="800" height="213" src="https://cdn.vox-cdn.com/thumbor/C1SdoDActSv8tPONx_OjwEobUjw=/0x0:1004x753/1200x800/filters:focal(0x0:1004x753)/cdn.vox-cdn.com/uploads/chorus_image/image/49523369/20150428-cloud-computing.0.jpg" width="320" /></a></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: left;">
<div style="text-align: justify;">
A <b>native cloud application </b>is an application that is developed for a cloud computing environment.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
There is no specific answer to the question "<i>what is a cloud-native application</i>" but different concepts that must be met. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
One of the most important in my opinion is the ability to <i>scale up and down</i> at a rapid rate. And this means that our applications cannot have any state on each of the servers since if one server goes down or is scaled down, then the state stored in that server will be lost.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
This is very well summarized at <a href="https://www.youtube.com/watch?v=osz-MT3AxqA">https://www.youtube.com/watch?v=osz-MT3AxqA</a> where it is explained with a shopping cart example. In monolith approach, you store the products of the shopping cart in a server session, if the server went down then all products of shopping cart were lost as well. In a <b>cloud-native app</b>, where server instances can be scaled up and down quickly, it is important to not have this stateful behavior on your services and design them to be stateless.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
There are different approaches to achieve this goal of implementing a <i>stateless architecture</i> but they can be summarized into two categories:</div>
<div style="text-align: justify;">
</div>
<ul>
<li>Use a distributed in-memory key/value data store like Infinispan.</li>
<li>Use a token which acts as a <i>session</i> between client and server using for example <b>JWT</b>.</li>
</ul>
<div style="text-align: justify;">
In this post, I am going to introduce you the later approach.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
From <a href="https://jwt.io/introduction/">https://jwt.io/introduction/</a> site:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<i><b>JSON Web Token (JWT)</b> is an open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. </i></div>
<div style="text-align: justify;">
<i><br /></i></div>
<div style="text-align: justify;">
<i>This information can be verified and trusted because it is digitally signed. <b>JWTs</b> can be signed using a secret using HMAC or a public/private key pair using RSA.</i></div>
<div style="text-align: justify;">
<i><br /></i></div>
<div style="text-align: justify;">
<i><b>JSON Web Tokens</b> consist of three Base64Url strings separated by dots which are: Header.Payload.Signature</i></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
So the idea basic idea for implementing stateless architecture on backend using <b>JWT</b> is the next one:</div>
<div style="text-align: justify;">
</div>
<ol>
<li>When the user adds the first product, the backend service generates a new <b>JWT</b> token with the product added and sent it back to the frontend.</li>
<li>When the user adds a new product, it sends the product to add and also the <b>JWT</b> token that was sent before by backend.</li>
<li>Then the backend verifies that the token has not been modified (verifying the signature), then it gets the products from <b>JWT</b> payload added previously and add the new one to the list. Finally, it creates a new token with previous and new products and it sent it back to the frontend.</li>
<li>The same process is repeated all the time.</li>
</ol>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-8-K6v25MYVs/WkyrRqoUdhI/AAAAAAAAB8g/8YGGjU5TqsA7_ITRwTKvEa7wNd18nrHlwCLcBGAs/s1600/Shoping%2BCart.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="531" data-original-width="564" height="376" src="https://2.bp.blogspot.com/-8-K6v25MYVs/WkyrRqoUdhI/AAAAAAAAB8g/8YGGjU5TqsA7_ITRwTKvEa7wNd18nrHlwCLcBGAs/s400/Shoping%2BCart.png" width="400" /></a></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
So as you can see, now it is not necessary to maintain any state or add any new database service on backend side, you just need to sent back and forward the <b>JWT</b> token with the products inside.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I have recorded a video of a simple shopping cart example where I show the stateless nature of the solution. It can be seen at:</div>
<div style="text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/xx93_YswRtk/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/xx93_YswRtk?feature=player_embedded" width="320"></iframe></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Also if you want to check the project that I used for recording you can take a look at <a href="https://github.com/lordofthejars/shop-jwt">https://github.com/lordofthejars/shop-jwt</a>.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Notice that this is just a simple post so you can get the basic idea. But you need to take into consideration next things to use it in production:</div>
<div style="text-align: justify;">
</div>
<ol>
<li>Use HTTPS instead of HTTP</li>
<li><b>JWT</b> just signs the token, if you want extra protection apart from HTTPS, use JWE to encrypt the payload of <b>JWT</b> token as well.</li>
<li>Fingerprinting the token to avoid any man-in-the-middle attack and use these parameters as authentication parameters for the token.</li>
<li><b>JWT</b> can be used for passing authentication and authorization things as well.</li>
</ol>
<div style="text-align: justify;">
You can watch my talk at JavaZone where I introduce some of these techniques:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<iframe allowfullscreen="" frameborder="0" height="360" mozallowfullscreen="" src="https://player.vimeo.com/video/138956646" webkitallowfullscreen="" width="640"></iframe> <a href="https://vimeo.com/138956646">Make your REST services attack proof - Alex Soto Bueno</a> from <a href="https://vimeo.com/javazone">JavaZone</a> on <a href="https://vimeo.com/">Vimeo</a>.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The good part of <b>JWT</b> approach is that it simplifies a lot the deployment of the service, you don't need to deploy or configure any other distributed database to share the content across the cluster, which minimizes the problems related to the network for communicating to the distributed database or misconfiguring of any of the nodes.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The drawback is that the client needs to be aware to receive and sent back the token and deal with it. In backend side, you need to sign and verify every token all the time.</div>
<div style="text-align: justify;">
<br />
Note that this approach might work in some cases and might get into some troubles in others (for example if there are parallel connections to backend all of them modifying the token). This post just shows an example of how I implemented this stateless thing in a project with specific requirements, but in other cases it might be wrong to do it. A real shopping cart implementation would have some problems, but for the sake of simplicity and having a business model that everyone undertands, I decided to implement it in this way.<br />
<br /></div>
<div style="text-align: justify;">
We keep learning,</div>
<div style="text-align: justify;">
Alex.</div>
<div style="text-align: justify;">
<blockquote class="tr_bq">
Turn it up, it's your favorite song (hey), Dance, dance, dance to the distortion, Turn it up (turn it up), keep it on repeat, Stumbling around like a wasted zombie (like a wasted zombie) (Chained to the Rhythm - Katy Perry)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=Um7pMggPnug">https://www.youtube.com/watch?v=Um7pMggPnug</a><br />
<br />
Follow me at <a href="https://twitter.com/alexsotob">https://twitter.com/alexsotob</a><br />
<br /></div>
</div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com13tag:blogger.com,1999:blog-19517292.post-69658769797789061702018-01-02T12:50:00.002+01:002018-01-18T08:31:05.353+01:00Writing end to end test for a microservices architecture<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.parksun.com/images/Bocce_web_boundary_lo_800.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://www.parksun.com/images/Bocce_web_boundary_lo_800.png" data-original-height="533" data-original-width="800" height="213" width="320" /></a></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: justify;">
UPDATE: I was not really sure if I should call this post with e2e tests since I knew it can get some confusions, but I couldn0t think about any other name. After reading <a href="https://martinfowler.com/bliki/IntegrationTest.html">https://martinfowler.com/bliki/IntegrationTest.html</a> I can be sure that what I am describing here is how to do <b>Narrow Integration Test</b>.<br />
<br />
One of the main aspects of microservices architecture is that the application is formed as a collection of loosely coupled services each one deployable independently and communicated each other with some kind of light protocol.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
It is because of microservices architecture is a distributed system that makes writing end to end tests really hard. Suppose next simple example provided by Red Hat as an example of microservices architecture (<a href="https://github.com/jbossdemocentral/coolstore-microservice">https://github.com/jbossdemocentral/coolstore-microservice</a>):</div>
<div style="text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://raw.githubusercontent.com/lordofthejars/coolstore-microservice/1.2.x/docs/images/arch-diagram.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="412" data-original-width="800" height="205" src="https://raw.githubusercontent.com/lordofthejars/coolstore-microservice/1.2.x/docs/images/arch-diagram.png" width="400" /></a></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Now suppose that you want to write an end to end test for <i>Cart Service</i>. You will quickly see that it is not easy at all, let me enumerate some of the reasons:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
</div>
<ul>
<li><i>Cart Service</i> needs to know how to boot up <i>Pricing Service</i>, <i>Catalog Service</i>, and <i>MongoDB</i> (and if you want to involve the front-end as well then <i>Coolstore GW</i> and <i>WebUI</i>).</li>
<li><i>Cart Service</i> needs to prepare some data (fixtures) for both of external services.</li>
<li>You communicate with services using a network. It might occur that some tests fail not because of a real failure but because of an infrastructure problem or because the other services have any bug. So the probability of these tests become flaky and start failing not because any changed introduced in current service is higher.</li>
<li>In more complex cases running these tests might be expensive, in terms of cost (deploying to the cloud), time (booting up all the infrastructure and services) and maintenance time.</li>
<li>Difficult to run them in developer machine, since you need all the pieces installed on the machine.</li>
</ul>
<br />
<div style="text-align: justify;">
For this reason, the end to end tests is not the best approach for testing a microservice, but you still need a way to test from the beginning to the end of the service.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
It is necessary to find a way to "simulate" these external dependencies without having to inject any mock object. What we need to do is cheat the service under test so it really thinks it is communicating with the real external services, when in reality it is not.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The method that allows us to do it is Service Virtualiztion. Service virtualization is a method to emulate the behavior of component applications such as API based.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
You can think about service virtualization as mocking approach you used to implement in OOP but instead of simulating at the object level, you simulate at the service level. It is mocking for the enterprise.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
There are a lot of service virtualization tools out there, but in my experience, in the JVM ecosystem, one of the tools that work better is <b>Hoverfly</b>.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Let's see how an "end-to-end" test looks like for Cart Service.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/f841b2dbe15984c572b3bc7615e3b8f7.js"></script>
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
This service is implemented using <b>Spring Boot</b>, so we are using Spring Boot Test framework. The important part here is that the URL where Catalog service is deployed is specified by using <i>CATALOG_ENDPOINT</i> property. And for this test, it is set to <i>catalog</i>.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The next important point is the Hoverfly class rule section. In that rule next things are specified:</div>
<div style="text-align: justify;">
</div>
<ol>
<li>An Http proxy is started before the test and all outgoing traffic from JVM is redirected to that proxy.</li>
<li>It records that when a request to host <i>catalog</i> is done and the path is <i>/api/products</i> it must return a success result with given json document.</li>
</ol>
<br />
<div style="text-align: justify;">
The test itself just uses <i>TestRestTemplate</i> (it is a rest client) and validates that you can add some elements to the cart.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Notice that you don't need to configure where the Http proxy is started or configure any port because <b>Hoverfly</b> automatically configures JVM network parameters so any network communication goes through Hoverfly proxy.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
So notice that now you don't need to know how to boot up Catalog service nor how to configure it with correct data.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
You are testing the whole service within its boundaries, from incoming messages to outgoing messages to other services, without mocking any internal element.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Probably you are wondering "What's happening in case of current service has also a dependency on a database server?"</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
In this case, you do as usual since the service itself knows which database server is using and the kind of data it requires, you only need to boot up the database server, populate required data (fixtures) and execute tests. For this scenario I suggest you using <b>Arquillian Cube Docker</b> to bootup database service from a Docker container so you don't need to install it on each machine you need to run tests and <b>Arquillian Persistence Extension</b> for maintaining the database into a known state.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
In next example of rating service, you can see briefly how to use them for persistence tests:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/f0a615341ad901fc1a1d70ff0aa8f519.js"></script>
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
With this approach, you are ensuring that all inner components of the service work together as expected and avoiding the flakiness nature of end to end tests in microservices.</div>
<div style="text-align: justify;">
<br />
So end to end tests in any microservice is not exactly the same of an end to end test in a monolith application, you are still testing the whole service, but just keeping a controlled environment, where test only depends on components within the boundary of service.<br />
<br /></div>
<div style="text-align: justify;">
How contract tests fit on this? Well actually everything showed here can be used in consumer and provider side of contract testing to avoid having to boot up any external service. In this way, as many authors conclude, if you are using contract tests, these are becoming the new end to end tests.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
You can see full projects where both tests are used at <a href="https://github.com/lordofthejars/coolstore-microservice/blob/a6fb45710f4830f77221ed3d47e808f7ef3f2790/cart-service/src/test/java/com/redhat/coolstore/CartServiceBoundaryTest.java">https://github.com/lordofthejars/coolstore-microservice/blob/a6fb45710f4830f77221ed3d47e808f7ef3f2790/cart-service/src/test/java/com/redhat/coolstore/CartServiceBoundaryTest.java</a> and <a href="https://github.com/lordofthejars/coolstore-microservice/blob/22779d9e70df2db244e6be694c7e35676fb1674f/rating-service/src/test/java/com/redhat/coolstore/rating/verticle/service/ApeCubeRatingServiceTest.java">https://github.com/lordofthejars/coolstore-microservice/blob/22779d9e70df2db244e6be694c7e35676fb1674f/rating-service/src/test/java/com/redhat/coolstore/rating/verticle/service/ApeCubeRatingServiceTest.java</a><br />
<br />
We keep learning,<br />
Alex.<br />
<blockquote class="tr_bq">
When every star falls from the sky, And every last heart in the world breaks, It's gonna be OK. (Ok - Robin Schulz)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=P9-4xHVc7uk">https://www.youtube.com/watch?v=P9-4xHVc7uk</a><br />
<br />
Follow me at <a href="https://twitter.com/alexsotob">https://twitter.com/alexsotob</a><br />
<br />
<br /></div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com14tag:blogger.com,1999:blog-19517292.post-71411788011388018312017-12-29T18:44:00.001+01:002017-12-29T18:52:31.303+01:00Java Champion<div class="separator" style="clear: both; text-align: center;">
<a href="https://cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/279734aa-8672-41f3-b463-eada17c58abe/File/9f2d0435773ba3ae6e1a0cb24a4ef4b6/java_champions_cmyk_copy.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="164" data-original-width="395" height="132" src="https://cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/279734aa-8672-41f3-b463-eada17c58abe/File/9f2d0435773ba3ae6e1a0cb24a4ef4b6/java_champions_cmyk_copy.jpg" width="320" /></a></div>
<div style="text-align: center;">
<br /></div>
<br />
<div style="text-align: justify;">
Before Christmas, I received the good news that I become a Java Champion. And then many memories came to my mind. My first computer, the ZX Spectrum that my parents bought me when I was 6, the first game I had, Desperado (<a href="http://www.worldofspectrum.org/infoseekid.cgi?id=0009331">http://www.worldofspectrum.org/infoseekid.cgi?id=0009331</a>) from TopoSoft. All programs that I copied from books in Basic language, all modifications I did and of course all the "pokers" that I copied from magazines to have infinite lives in games.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
After that, it started the PC era, where I remember creating many "bat" files for having different memory configurations for playing games, editing saved files with Hex editors to have millions of resources and the appearance of Turbo C in my computer.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Finally, the internet era, that for me started in 1997, and my first browser which it was Netscape. What good old time, and then my first web page and JavaScript.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
And this moves me towards the key point, I think it was at 1999 or 2000, I bought a magazine which it came with a CD-ROM with Java 1.2 there, I installed and I started programming in Java without IDE until today.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Probably this is the life of many of developers that were born in the 80s. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
But this is only "scientific" facts, not feelings. These days I have been thinking about the feelings I had, what ruled to me all this time, and it came to my mind the opening theme of one of my favorite TV program called "This is Opera". Exactly the same words can be applied in my case for programming/computing/Java.</div>
<div style="text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/tGulfVpncTQ/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/tGulfVpncTQ?feature=player_embedded" width="320"></iframe></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I cannot finish without thanking all the people who I met over the globe during conferences, starting with Dan Allen (the first one I met at a conference), all Arquillian team, Tomitribe team with David Belvins at head), Asciidoctor community, Barcelona JUG guys and a lot of people, organizers, attendees, .... every one thank you so much for being there.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Of course, all the workmates I had in University (La Salle), in Aventia, Grifols (thanks to Erytra team, I enjoyed so much during all years I was there), Everis, Scytl, CloudBees (LLAP to Jenkins) and finally Red Hat, a company that I really love and I feel like home.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Last but not least my family who has been there all the time.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Next year I will continue posting more posts, contributing more to open source projects and of course going to speak to as many events as possible.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
We keep learning,</div>
<div style="text-align: justify;">
Alex.</div>
<blockquote class="tr_bq" style="text-align: justify;">
Caminante son tus huellas el camino y nada más: caminante, no hay camino se hace el camino al andar (Caminante No Hay Camino - Joan Manuel Serrat)</blockquote>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Music: <a href="https://www.youtube.com/watch?v=2DA3pRht2MA">https://www.youtube.com/watch?v=2DA3pRht2MA</a></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Follow me at <a href="https://twitter.com/alexsotob">https://twitter.com/alexsotob</a></div>
<div style="text-align: justify;">
<br /></div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com7tag:blogger.com,1999:blog-19517292.post-13387004409623847862017-10-25T16:46:00.004+02:002017-10-25T16:46:51.330+02:00Adding Chaos on OpenShift cluster<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.altarandthrone.com/wp-content/uploads/2016/05/205057-chaos.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://www.altarandthrone.com/wp-content/uploads/2016/05/205057-chaos.jpg" data-original-height="535" data-original-width="800" height="214" width="320" /></a></div>
<br />
<div style="text-align: justify;">
<b><a href="https://github.com/gaia-adm/pumba">Pumba</a></b> is a chaos testing tool for Docker and <b>Kubernetes</b> (hence <b>OpenShift</b> too). It allows you to add some chaos on containers instances such as stopping, killing or removing containers randomly. But it also adding some network chaos such as delaying, packet loss or re-ordering.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
You can see in next screen recording how to add Pumba on OpenShift cluster and add some chaos on it.</div>
<br />
<iframe allowfullscreen="" frameborder="0" gesture="media" height="315" src="https://www.youtube.com/embed/kA0P-V2JPTA?ecver=1" width="560"></iframe>
<br />
<br />
<div style="text-align: justify;">
The security calls that you need to run before deploying <b>Pumba</b> are the next ones:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/eab3db728df66ac44ce0d8d31b6224dc.js"></script>
Also the Kubernetes script to deploy <b>Pumba</b> in <b>OpenShift</b> is:</div>
<div style="text-align: justify;">
<br /></div>
<script src="https://gist.github.com/lordofthejars/3a342bc8ac7e253ccec35b7ff69f56a1.js"></script>
<br />
<div style="text-align: justify;">
I have sent a PR to upstream <b>Pumba</b> project with this file but meanwhile is not accepted you can use it this one.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I'd like to say thank you to Slava Semushin and Jorge Morales for helping me on understanding the <b>OpenShift</b> security model.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
We keep learning,</div>
<div style="text-align: justify;">
Alex.</div>
<blockquote class="tr_bq">
Ce joli rajolinet, que les oques tonifique, si le fique en une pique, mantindra le pompis net (El baró de Bidet - La Trinca)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=4JWIbKGe4gA">https://www.youtube.com/watch?v=4JWIbKGe4gA</a><br />
<br />
Follow me at <a href="https://twitter.com/alexsotob">https://twitter.com/alexsotob</a><br />
<div>
<br /></div>
<br />Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com7tag:blogger.com,1999:blog-19517292.post-31853661200580481922017-10-24T17:10:00.003+02:002017-10-24T17:10:40.703+02:00Testing Code that requires a mail server<div class="separator" style="clear: both; text-align: center;">
<a href="http://iconshow.me/media/images/Mixed/line-icon/png/512/mail-512.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://iconshow.me/media/images/Mixed/line-icon/png/512/mail-512.png" data-original-height="512" data-original-width="512" height="200" width="200" /></a></div>
<div style="text-align: justify;">
Almost all applications has one common requirement, they need to send an email notifying something to a registered user. It might be an invoice, a confirmation of an action or a password recovery. How to test this use case might be challenging, using mocks and stubs are ok for unit tests, but having a component test that tests the whole stack. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
In this post I am going to show you how <b>Docker</b> and <b>MailHog</b>, can help you on testing this part of code.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<b><a href="https://github.com/mailhog/MailHog">MailHog</a></b> is super simple SMTP server for email testing tool for developers:</div>
<ul>
<li style="text-align: justify;">Configure your application to use MailHog for SMTP delivery</li>
<li style="text-align: justify;">View messages in the web UI, or retrieve them with the JSON API</li>
<li style="text-align: justify;">Optionally release messages to real SMTP servers for delivery</li>
<li style="text-align: justify;">Docker image with MailHog installed</li>
</ul>
<div style="text-align: justify;">
Notice that since you can retrieve any message sent to mail server using JSON API, it makes really simple to validate if the message has been really delivered and of course assert on any of message fields.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<a href="http://arquillian.org/arquillian-cube/"><b>Arquillian Cube</b></a> is an <b>Arquillian</b> extension that can be used to manager <b>Docker</b> containers in your tests. To use it you need a <b>Docker</b> daemon running on a computer (it can be local or not), but probably it will be at local.</div>
<br />
<div style="text-align: justify;">
<b>Arquillian Cube</b> offers three different ways to define container(s):</div>
<ul>
<li style="text-align: justify;">Defining a docker-compose file.</li>
<li style="text-align: justify;">Defining a Container Object.</li>
<li style="text-align: justify;">Using Container Object DSL.</li>
</ul>
<div style="text-align: justify;">
In this example I am going to show you Container Object DSL approach, but any of the others works as well.</div>
<div>
<div style="text-align: justify;">
<br /></div>
</div>
<div>
<div style="text-align: justify;">
To use Container Object DSL you simply need to instantiate the <i>ContainerDslRule</i> (in case you are using JUnit Rules) or use <b>Arquillian</b> runner in case of using JUnit, TestNG or Spock. You can read more about Container Object DSL at <a href="http://arquillian.org/arquillian-cube/#_arquillian_cube_and_container_object_dsl">http://arquillian.org/arquillian-cube/#_arquillian_cube_and_container_object_dsl</a></div>
</div>
<div>
<br /></div>
<div>
<div style="text-align: justify;">
As an example of definition of Redis container:</div>
</div>
<div>
<br /></div>
<div>
<script src="https://gist.github.com/lordofthejars/e2a3eeb683019f24d2d95c45ef9e3e46.js"></script>
</div>
<div>
<br /></div>
<div>
When running this test, Redis Docker image is started, tests are executed and finally the Docker instance is stopped.<br />
<br />
<div style="text-align: justify;">
So let's see how to do the same but instead of Redis using <b>Mailhog Docker</b> image.</div>
<div style="text-align: justify;">
It is important to notice that <i>ContainerDslRule</i> is a generic class that can be extended to become more specific to a concrete use case. And this is what we are going to do for <b>Mailhog</b>.</div>
<br />
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/d20406be8cb38fd59001fb3f1a37cdc9.js"></script>
First of all we need to create a class extending from <i>ContainerDslRule</i>, so everything is still a JUnit rule, but a customized one. Then we create a factory method which creates the <i>MailhogContainer</i> object, setting the image name and the binding ports. Finally an assertion method is used to connect to Rest API of Mailhog server to check if there is any message with given subject.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Then we can write a test using this new rule.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/3d2a766dddc9d1999e3c2b802ea5974f.js"></script>
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
This test just configures <i>MailService</i> class with <b>Docker</b> container configuration, it sends an email and finally it delegates to container object to validate if the email has been received or not.<br />
<br />
Notice that putting everything into an object makes this object reusable in other tests and even in other projects. You can create an independent project with all your custom developed container objects and just reuse them importing them as test jar in hosted project.<br />
<br />
Code: <a href="https://github.com/lordofthejars/mailtest">https://github.com/lordofthejars/mailtest</a><br />
<br />
We keep learning,<br />
Alex.<br />
<blockquote class="tr_bq">
'Cause I'm kind of like Han Solo always stroking my own wookie, I'm the root of all that's evil yeah but you can call me cookie (Fire Water Burn - Bloodhound Gang)</blockquote>
</div>
<div style="text-align: justify;">
Music: <a href="https://www.youtube.com/watch?v=Adgx9wt63NY">https://www.youtube.com/watch?v=Adgx9wt63NY</a><br />
<br />
Follow me at <a href="https://twitter.com/alexsotob">https://twitter.com/alexsotob</a><br />
<br /></div>
<br />
<br /></div>
<div>
<br /></div>
<br />Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com7tag:blogger.com,1999:blog-19517292.post-67024306356101790892017-09-19T11:17:00.000+02:002017-09-19T11:17:24.567+02:00Testing code that uses Java System Properties<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.scottphillips.com/files/2011/01/Configuration.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://www.scottphillips.com/files/2011/01/Configuration.png" data-original-height="200" data-original-width="200" height="200" width="200" /></a></div>
<div class="separator" style="clear: both; text-align: justify;">
Sometimes your code uses any <b>Java System Property</b> to configure itself. Usually these classes are configuration classes that can get properties from different sources and one of valid one is using Java System Properties.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
The problem is how to write tests for this code? Obviously to maintain your tests isolated you need to set and unset them for each test, restoring old value if for example you are dealing with a global property which is already set before executing tests, and of course in case of an error do not forget to unset/restore the value. So arrived at this point the test might look like:</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<script src="https://gist.github.com/lordofthejars/4588844e431ff58152a97a3b9b751515.js"></script>
</div>
<div class="separator" style="clear: both; text-align: justify;">
Notice that this structure should be repeated for each test method that requires an specific system property, so this structure becomes a boilerplate code.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
To avoid having to repeat this code over and over again, you can use <b>System Rules</b> project which is a collection of JUnit rules for testing code that uses <i>java.lang.System</i> </div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
The first thing you need to do is add <b>System Rules</b> dependency as test scope in your build tool, which in this case is <i>com.github.stefanbirkner:system-rules:1.16.0.</i></div>
<div class="separator" style="clear: both; text-align: justify;">
<i><br /></i></div>
<div class="separator" style="clear: both; text-align: justify;">
Then you need to register the JUnit Rule into your test class.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<script src="https://gist.github.com/lordofthejars/cf5f53c0556faa1a8b6b8b9393d448c4.js"></script></div>
<div class="separator" style="clear: both; text-align: justify;">
In this case, you can freely set a System property in your test, the JUnit rule will take care of saving and restoring the System property.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
That's it, really easy, no boilerplate code and help you maintaining your tests clean.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Just one notice, these kind of tests are not thread safe, so you need to be really cautious when you use for example surefire plugin with forks. One possible way to avoiding this is by using <i>net.jcip.annotations.NotThreadSafe.class</i> capabilities.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
We keep learning,</div>
<div class="separator" style="clear: both; text-align: justify;">
Alex</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<blockquote class="tr_bq" style="clear: both; text-align: justify;">
Polly wants a cracker, I think I should get off her first, I think she wants some water, To put out the blow torch (Polly - Nirvana)</blockquote>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Music: <a href="https://www.youtube.com/watch?v=Qd3rpF9v-1o">https://www.youtube.com/watch?v=Qd3rpF9v-1o</a></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Follow me at <a href="https://twitter.com/alexsotob">https://twitter.com/alexsotob</a></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com3tag:blogger.com,1999:blog-19517292.post-83331391255667204432017-06-27T12:34:00.000+02:002017-06-27T14:52:26.674+02:00Lifecycle of JUnit 5 Extension Model<div class="separator" style="clear: both; text-align: center;">
<a href="http://junit.org/junit4/images/junit5-banner.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://junit.org/junit4/images/junit5-banner.png" data-original-height="244" data-original-width="800" height="60" width="200" /></a></div>
<br />
<div style="text-align: justify;">
<b>JUnit5</b> final release is around the corner (currently it is M4), and I have started playing with it a bit on how to write extensions.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
In <b>JUnit5</b>, instead of dealing with <i>Runners</i>, <i>Rules</i>, <i>ClassRules</i> and so on, you've got a single <b>Extension API</b> to implement your own extensions.</div>
<br />
<div style="text-align: justify;">
<b>JUnit5</b> provides several interfaces to hook in its lifecycle. For example you can hook to <i>Test Instance Post-processing </i>to invoke custom initialization methods on the test instance, or <i>Parameter Resolution</i> for dynamically resolving test method parameters at runtime. And of course the typical ones like hooking before all tests are executed, before a test is executed, after a test is executed and so on so far, a complete list can be found at <a href="http://junit.org/junit5/docs/current/user-guide/#extensions-lifecycle-callbacks">http://junit.org/junit5/docs/current/user-guide/#extensions-lifecycle-callbacks</a></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
But in which point of the process it is executed each of them? To test it I have just created an extension that implements all interfaces and each method prints who is it.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/e86cede969e7012cc4ec257d4a2b58da.js"></script>
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Then I have created a <b>JUnit5</b> Test suite containing two tests:<br />
<br />
<script src="https://gist.github.com/lordofthejars/af4155804208ad7efb7310c67d1995fe.js"></script>
So after executing this suite, what it is the output? Let's see it. Notice that for sake of readability I have added some callouts on terminal output.<br />
<br/>
<script src="https://gist.github.com/lordofthejars/d321a1deca9d58f3c6f346516a171422.js"></script>
<br />
<b><1></b> First test that it is run is <i>AnotherLoggerExtensionTest</i>. In this case there is only one simple test, so the lifecycle of extension is <i>BeforeAll</i>, <i>Test Instance-Post-Processing</i>, <i>Before Each</i>, <i>Before Test Execution</i>, then the test itself is executed, and then all <i>After</i> callbacks.<br />
<br />
<b><2> </b>Then the <i>LoggerExtensionTest</i> is executed. First test is not a parametrized test, so events related to <i>parameter resolution</i> are not called. Before the test method is executed, <i>test instance post-processing</i> is called, and after that all before events are thrown. Finally the test is executed with all after events.<br />
<br />
<b><3></b> Second test contains requires a parameter resolution. <i>Parameter resolvers</i> are run after <i>Before</i> events and before executing the test itself.<br />
<br />
<b><4></b> Last test throws an exception. <i>Test Execution Exception</i> is called after test is executed but before <i>After</i> events.<br />
<br />
Last thing to notice is that <i>BeforeAll</i> and <i>AfterAll</i> events are executed per test class and not suite.<br />
<br />
The <b>JUnit</b> version used in this example is <i>org.junit.jupiter:junit-jupiter-api:5.0.0-M4</i><br />
<br />
We keep learning,<br />
Alex<br />
<blockquote class="tr_bq">
That's why we won't back down, We won't run and hide, 'Cause these are the things we can't deny, I'm passing over you like a satellite (Satellite - Rise Against)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=6nQCxwneUwA">https://www.youtube.com/watch?v=6nQCxwneUwA</a><br />
<br />
Follow me at <a href="http://www.twitter.com/alexsotob"><img alt="" src="http://www.logospike.com/wp-content/uploads/2014/11/Twitter_logo-12.png" title="" width="10%" /></a><br />
<br /></div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com3tag:blogger.com,1999:blog-19517292.post-77169154915244343672017-06-23T16:53:00.002+02:002017-06-23T16:53:52.657+02:00Test AWS cloud stack offline with Arquillian and LocalStack<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-LFNzAzH690Q/WUztnCVPzlI/AAAAAAAABqI/mrTxnS7piGAtJhiQCgeX--G8U3FOqlUEwCLcBGAs/s1600/330981.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="200" data-original-width="200" height="150" src="https://4.bp.blogspot.com/-LFNzAzH690Q/WUztnCVPzlI/AAAAAAAABqI/mrTxnS7piGAtJhiQCgeX--G8U3FOqlUEwCLcBGAs/s200/330981.png" width="150" /></a><a href="https://2.bp.blogspot.com/-MWmlfP23wFg/WUztnORWBkI/AAAAAAAABqQ/YmQ0zq7WgOccltjZbTpn3XceA1WI4XmUQCLcBGAs/s1600/aws-graphic.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="273" data-original-width="459" height="118" src="https://2.bp.blogspot.com/-MWmlfP23wFg/WUztnORWBkI/AAAAAAAABqQ/YmQ0zq7WgOccltjZbTpn3XceA1WI4XmUQCLcBGAs/s200/aws-graphic.jpg" width="200" /></a><a href="https://4.bp.blogspot.com/-ATTvPE9JIm4/WUztnGtg9CI/AAAAAAAABqM/jiWtyoO-5T0HXWOqEzjS2t5Xtux4_zy6ACLcBGAs/s1600/localstack.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="157" data-original-width="300" height="104" src="https://4.bp.blogspot.com/-ATTvPE9JIm4/WUztnGtg9CI/AAAAAAAABqM/jiWtyoO-5T0HXWOqEzjS2t5Xtux4_zy6ACLcBGAs/s200/localstack.png" width="200" /></a></div>
<br />
<div style="text-align: justify;">
When you are building your applications on <b>AWS cloud stack</b> (such as DynamoDB, S3, ...), you need to write tests against these components. The first idea you might have is to have one environment for production and another one for testing, and run tests against it.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
This is fine for integration tests, deployment tests, end to end tests or performance tests, but for component tests it will be faster if you could run <b>AWS cloud stack</b> locally and offline.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<a href="https://github.com/atlassian/localstack"><b>Localstack</b></a> provides this feature. It provides a fully functional local <b>AWS cloud stack</b> so you can develop and test your cloud applications offline.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<b>Localstack</b> comes with different ways to start all stack, but the easiest one is by using Docker image. So if you run <i>atlassianlabs/localstack</i> then you get the stack up and running with next configuration:</div>
<ul>
<li>API Gateway at http://localhost:4567</li>
<li>Kinesis at http://localhost:4568</li>
<li>DynamoDB at http://localhost:4569</li>
<li>DynamoDB Streams at http://localhost:4570</li>
<li>Elasticsearch at http://localhost:4571</li>
<li>S3 at http://localhost:4572</li>
<li>Firehose at http://localhost:4573</li>
<li>Lambda at http://localhost:4574</li>
<li>SNS at http://localhost:4575</li>
<li>SQS at http://localhost:4576</li>
<li>Redshift at http://localhost:4577</li>
<li>ES (Elasticsearch Service) at http://localhost:4578</li>
<li>SES at http://localhost:4579</li>
<li>Route53 at http://localhost:4580</li>
<li>CloudFormation at http://localhost:4581</li>
<li>CloudWatch at http://localhost:4582</li>
</ul>
<div style="text-align: justify;">
So the next question is how do you automate all the process of starting the container, run the tests and finally stop everything and make it portable, so you don't need to worry if you are using Docker in Linux or MacOS? The answer is using <b>Arquillian Cube</b>.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<div>
<b><a href="http://arquillian.org/arquillian-cube">Arquillian Cube</a></b> is an <i>Arquillian</i> extension that can be used to manager Docker containers in your tests. To use it you need a Docker daemon running on a computer (it can be local or not), but probably it will be at local.</div>
<div>
<br /></div>
<div>
<b>Arquillian Cube</b> offers three different ways to define container(s):</div>
<div>
<ul>
<li>Defining a <i>docker-compose</i> file.</li>
<li>Defining a Container Object.</li>
<li>Using Container Object DSL.</li>
</ul>
</div>
<div>
In this example I am going to show you Container Object DSL approach, but any of the others works as well.</div>
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The first thing you need to do is add Arquillian and <b>Arquillian Cube</b> dependencies on your build tool.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/f60f7ed54c9c0f0e9f055873b7350e5d.js"></script>
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Then you can write the test which in this case tests that you can create a bucket and add some content using the S3 instance started in Docker host:<br />
<br />
<script src="https://gist.github.com/lordofthejars/9ef04ac606dc91880e6d12015d431286.js"></script>
Important things to take into consideration:<br />
<ol>
<li>You annotate your test with Arquillian runner.</li>
<li>Use <i>@DockerContainer</i> annotation to attribute used to define the container.</li>
<li>Container Object DSL is just a DSL that allows you to configure the container you want to use. In this case the <i>localstack</i> container with required port binding information.</li>
<li>The test just connects to Amazon S3 and creates a bucket and stores some content.</li>
</ol>
<div>
Nothing else is required. When you run this test, <b>Arquillian Cube</b> will connect to installed Docker (Machine) host and start the <i>localstack</i> container. When it is up and running and services are able to receive requests, the tests are executed. After that container is stopped and destroyed.</div>
<div>
<br /></div>
<div>
<b>TIP1</b>: If you cannot use Arquillian runner you can also use a JUnit Class Rule to define the container as described here <a href="http://arquillian.org/arquillian-cube/#_junit_rule">http://arquillian.org/arquillian-cube/#_junit_rule</a></div>
<div>
<br /></div>
<div>
<b>TIP2:</b> If you are planning to use <i>localstack</i> in the whole organization, I suggest you to use <i>Container Object</i> approach instead of DSL because then you can pack the <i>localstack</i> Container Object into a jar file and import in all projects you need to use it. You can read at <a href="http://arquillian.org/arquillian-cube/#_arquillian_cube_and_container_object">http://arquillian.org/arquillian-cube/#_arquillian_cube_and_container_object</a></div>
<br />
So now you can write tests for your application running on <b>AWS cloud </b>without having to connect to remote hosts, just using local environment.<br />
<br />
We keep learning,<br />
Alex<br />
<blockquote class="tr_bq">
Tú, tú eres el imán y yo soy el metal , Me voy acercando y voy armando el plan , Solo con pensarlo se acelera el pulso (Oh yeah) (Despacito - Luis Fonsi)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=kJQP7kiw5Fk">https://www.youtube.com/watch?v=kJQP7kiw5Fk</a><br />
<br /></div>
<div style="text-align: justify;">
<br /></div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com4tag:blogger.com,1999:blog-19517292.post-76523507808436901792017-06-22T11:29:00.000+02:002017-06-22T11:29:38.884+02:00Vert.X meets Service Virtualization with Hoverfly<div class="separator" style="clear: both; text-align: center;">
<a href="http://vertx.io/vertx2/logo-white-big.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://vertx.io/vertx2/logo-white-big.png" data-original-height="127" data-original-width="422" height="60" width="200" /></a><a href="https://raw.githubusercontent.com/SpectoLabs/hoverfly/master/hoverfly_logo.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="199" data-original-width="650" height="97" src="https://raw.githubusercontent.com/SpectoLabs/hoverfly/master/hoverfly_logo.png" width="320" /></a></div>
<br />
<div style="text-align: justify;">
<b><br /></b></div>
<div style="text-align: justify;">
<b>Service Virtualization</b> is a technique using to emulate the behaviour of dependencies of component-based applications.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<b>Hoverfly</b> is a service virtualisation tool written in Go which allows you to emulate HTTP(S) services. It is a proxy which responds to HTTP(S) requests with stored responses, pretending to be it’s real counterpart.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<b>Hoverfly Java</b> is a wrapper around <b>Hoverfly</b>, that let's you use it in Java world. It provides a native Java DSL to write expectations and a <b>JUnit</b> rule to use it together with <b>JUnit</b>.<br />
<br />
But apart from being able to program expectations, you can also use <b>Hoverfly</b> to capture traffic between both services (in both cases are real services) and persist it.<br />
<br />
Then in next runs <b>Hoverfly</b> will use these persisted scripts to emulate traffic and not touch the remote service. In this way, instead of programming expectations, which means that you are programming how you understand the system, you are using real communication data.<br />
<br />
This can be summarised in next figures:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-Lr8ytwOcxx8/WUpXmpLmyTI/AAAAAAAABnQ/wQt5OaGnz5MoTvJkPEWFJEoEoVwpQFdPwCLcBGAs/s1600/Screen%2BShot%2B2017-06-21%2Bat%2B13.24.22.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="474" data-original-width="986" height="153" src="https://4.bp.blogspot.com/-Lr8ytwOcxx8/WUpXmpLmyTI/AAAAAAAABnQ/wQt5OaGnz5MoTvJkPEWFJEoEoVwpQFdPwCLcBGAs/s320/Screen%2BShot%2B2017-06-21%2Bat%2B13.24.22.png" width="320" /></a></div>
<br />
First time the output traffic is sent though <b>Hoverfly</b> proxy, it is redirected to real service and it generates a response. Then when the response arrives to proxy, both request and response are stored, and the real response is sent back to caller.<br />
<br />
Then in next calls of same method:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-gsDVMn5ByoU/WUpZOyC9F0I/AAAAAAAABnc/4Ns8qThr2B06rIjWu1D4EzAtyXXkBG1tQCLcBGAs/s1600/Screen%2BShot%2B2017-06-21%2Bat%2B13.24.29.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="470" data-original-width="984" height="152" src="https://1.bp.blogspot.com/-gsDVMn5ByoU/WUpZOyC9F0I/AAAAAAAABnc/4Ns8qThr2B06rIjWu1D4EzAtyXXkBG1tQCLcBGAs/s320/Screen%2BShot%2B2017-06-21%2Bat%2B13.24.29.png" width="320" /></a></div>
<br />
The output traffic of Service A is still sent though <b>Hoverfly</b> proxy, but now the response is returned from previous stored responses instead of redirecting to real service.<br />
<br />
So, how to connect from HTTP client of Service A to <b>Hoverfly</b> proxy? The quick answer is nothing.<br />
<br />
<b>Hoverfly</b> just overrides Java network system properties (<a href="https://docs.oracle.com/javase/7/docs/api/java/net/doc-files/net-properties.html">https://docs.oracle.com/javase/7/docs/api/java/net/doc-files/net-properties.html</a>) so you don't need to do anything, all communications (independently of the host you put there) from HTTP client will go through <b>Hoverfly</b> proxy.<br />
<br />
The problem is what's happening if the API you are using as HTTP client does not honor these system properties? Then obviously all outgoing communications will not pass thorough proxy.<br />
<br />
One example is <b>Vert.X</b> and its HTTP client <i>io.vertx.rxjava.ext.web.client.WebClient</i>. Since WebClient does not honor these properties, you need to configure the client properly in order to use <b>Hoverfly</b>.<br />
<br />
The basic step you need to do is just configure <i>WebClient</i> with proxy options that are set as system properties.<br />
<br />
<script src="https://gist.github.com/lordofthejars/14abf7622164979fb1d44e0222305818.js"></script>
Notice that the only thing that it done here is just checking if system properties regarding network proxy has been configured (by <b>Hoverfly</b> Java) and if it is the case just create a <b>Vert.X</b> <i>ProxyOptions</i> object to configure the HTTP client.<br />
<br />
With this change, you can write tests with <b>Hoverfly</b> and <b>Vert.X </b>without any problem:<br />
<br />
<script src="https://gist.github.com/lordofthejars/46b194870e4b295c95278b9e46070439.js"></script>
In previous example <b>Hoverfly</b> is used as in simulate mode and the request/response definitions comes in form of DSL instead of an external JSON script.<br />
Notice that in this case you are programming that when a request by current service (<i>VillainsVerticle</i>), is done to host <i>crimes</i> and port <i>9090</i>, using <i>GET</i> HTTP method at <i>/crimes/Gru</i> then the response is returned. For sake of simplicity of current post this method is enough.<br />
<br />
You can see source code at <a href="https://github.com/arquillian-testing-microservices/villains-service">https://github.com/arquillian-testing-microservices/villains-service</a> and read about <b>Hoverfly</b> Java at <a href="http://hoverfly-java.readthedocs.io/en/latest/">http://hoverfly-java.readthedocs.io/en/latest/</a><br />
<br />
We keep learning,<br />
Alex<br />
<blockquote class="tr_bq">
No vull veure't, vull mirar-te. No vull imaginar-te, vull sentir-te. Vull compartir tot això que sents. No vull tenir-te a tu: vull, amb tu, tenir el temps. (Una LLuna a l'Aigua - Txarango)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=BeH2eH9iPw4">https://www.youtube.com/watch?v=BeH2eH9iPw4</a></div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com1tag:blogger.com,1999:blog-19517292.post-90501843524592895302017-05-24T15:47:00.000+02:002017-05-24T15:47:46.736+02:00Deploying Docker Images to OpenShift<div style="text-align: justify;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://www.safaribooksonline.com/library/view/getting-started-with/9781491900468/images/gsos_0101.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="192" src="https://www.safaribooksonline.com/library/view/getting-started-with/9781491900468/images/gsos_0101.png" width="400" /></a></div>
<br /></div>
<div style="text-align: justify;">
<b><br /></b>
<b>OpenShift</b> is RedHat's cloud development Platform as a Service (<i>PaaS</i>). It uses <b>Kubernetes</b> as container orchestration (so you can use <b>OpenShift</b> as <b>Kubernetes</b> implementation), but providing some features missed in <b>Kubernates</b> such as automation of the build process of the containers, health management, dynamic provision storage or multi-tenancy to cite a few.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
In this post I am going to explain how you can deploy a <b>Docker</b> image from <i>Docker Hub</i> into an <b>OpenShift</b> instance. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
It is important to note that <b>OpenShift</b> offers other ways to create and deploy a container into its infrastructure, you can read more about at <a href="https://docs.openshift.com/enterprise/3.2/dev_guide/builds.html">https://docs.openshift.com/enterprise/3.2/dev_guide/builds.html</a> but as read in previous paragraph, in this case I am going to show you how to deploy already created <b>Docker</b> images from <i>Docker Hub</i>.</div>
<br />
<div style="text-align: justify;">
First thing to do is create an account in <a href="https://www.openshift.com/">OpenShift Online</a>. It is free and for the sake of this post is enough. Of course you can use any other <b>OpenShift</b> approach like<b> OpenShift Origin</b>.</div>
<br />
After that you need to login into <b>OpenShift</b> cluster. In case of <b>OpenShift Online</b> using the token provided:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">oc login https://api.starter-us-east-1.openshift.com --token=xxxxxxx</span><br />
<br />
Then you need to create a new project inside <b>OpenShift</b>.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">oc new-project villains</span><br />
<br />
<div style="text-align: justify;">
You can understand a project as a <b>Kubernetes</b> namespace with additional features. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Then let's create a new application within previous project based on a <b>Docker</b> image published at <b>Docker Hub</b>. This example is a VertX application where you can get crimes from several fictional villains from Lex Luthor to Gru.<br />
<br /></div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">oc new-app lordofthejars/crimes:1.0 --name crimes</span><br />
<br />
In this case a new app called <i>crimes</i> is created based on <i>lordofthejars/crimes:1.0</i> image. After running previous command, a new pod running previous image + a service + a replication controller is created.<br />
<br />
After that we need to create a route so the service is available to public internet.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">oc expose svc crimes --name=crimeswelcome</span><br />
<br />
<div style="text-align: justify;">
The last step is just get the version of the service from the browser, in my case was: <i>http://crimeswelcome-villains.1d35.starter-us-east-1.openshiftapps.com/version</i> notice that you need to change the public host with the one generated by your router and then append <i>version</i>. You can find the public URL by going to <b>OpenShift</b> dashboard, at top of pods definition.</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-OSC1BM68-MU/WSRNkWSVsEI/AAAAAAAABio/wIiyLt11_2sblPncm_qceGRW9awlIXGZgCLcB/s1600/console_navigation.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="427" src="https://1.bp.blogspot.com/-OSC1BM68-MU/WSRNkWSVsEI/AAAAAAAABio/wIiyLt11_2sblPncm_qceGRW9awlIXGZgCLcB/s640/console_navigation.png" width="640" /></a></div>
<br />
<div style="text-align: justify;">
Ok now you'll get a <i>1.0</i> which is the version we have deployed. Now suppose you want to update to next version of the service, to version 1.1, so you need to run next commands to deploy next version of crimes service container, which is pushed at <b>Docker Hub</b>.</div>
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">oc import-image crimes:1.1 --from=lordofthejars/crimes:1.1</span><br />
<br />
<div style="text-align: justify;">
With previous command you are configuring internal <b>OpenShift</b> Docker Registry with next <b>Docker</b> image to release.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Then let's prepare the application so when next rollout command is applied, the new image is deployed:</div>
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">oc patch dc/crimes -p '{"spec": { "triggers":[ {"type": "ConfigChange", "type": "ImageChange" , "imageChangeParams": {"automatic": true, "containerNames":["crimes"],"from": {"name":"crimes:1.1"}}}]}}'</span><br />
<br />
And finally you can do the rollout of the application by using:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">oc rollout latest dc/crimes</span><br />
<br />
<div style="text-align: justify;">
After a few seconds you can go again to <i style="text-align: justify;">http://crimeswelcome-villains.1d35.starter-us-east-1.openshiftapps.com/version</i><span style="text-align: justify;"> (of course change the host with your host)</span><i style="text-align: justify;"> </i><span style="text-align: justify;">and the version you'll get is 1.1.</span></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Finally what's happening if this new version contains a bug, and you want to do a rollback of the deployment to previous version? Easy just run next command:</div>
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">oc rollback crimes-1</span><br />
<br />
<div style="text-align: justify;">
And previous version is going to be deployed again, so after a few seconds you can go again to <i>/version </i>and you'll see 1.0 version again.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Finally if you want to delete the application to have a clean cluster run:</div>
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">oc delete all --all</span>
<br />
<br />
<div style="text-align: justify;">
So as you can see, it is really easy to deploy to deploy container images from <b>Docker Hub</b> to <b>OpenShift</b>. Notice that there are other ways to deploy our application into <b>OpenShift</b> (<a href="https://docs.openshift.com/enterprise/3.2/dev_guide/builds.html" style="text-align: justify;">https://docs.openshift.com/enterprise/3.2/dev_guide/builds.html</a>), in this post I have just shown you one.</div>
<br />
Commands: <a href="https://gist.github.com/lordofthejars/9fb5f08e47775a185a9b1f80f4af7aff">https://gist.github.com/lordofthejars/9fb5f08e47775a185a9b1f80f4af7aff</a><br />
<br />
We keep learning,<br />
Alex.<br />
<blockquote class="tr_bq">
Yo listen up here's a story, About a little guy that lives in a blue world, And all day and all night and, everything he sees is just blue, Like him inside and outside (Blue - Eiffel 65)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=68ugkg9RePc">https://www.youtube.com/watch?v=68ugkg9RePc</a><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span>Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com3tag:blogger.com,1999:blog-19517292.post-73156273938847561062017-05-19T12:30:00.001+02:002017-05-19T12:30:12.463+02:00Running Parallel Tests in Docker<div style="text-align: justify;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://www.allaboutcircuits.com/uploads/articles/Parallel-Resistance-Calculator.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="95" src="https://www.allaboutcircuits.com/uploads/articles/Parallel-Resistance-Calculator.jpg" width="200" /></a></div>
<br />
<br /></div>
<div style="text-align: justify;">
Sometimes when you are running your tests on your CI environment, you want to run tests in parallel. This parallelism is programmed in build tool such as <i>Maven</i> or <i>Gradle</i> or by using <i>Jenkins</i> plugin. </div>
<br />
<div style="text-align: justify;">
If you are using <b>Docker</b> as a testing tool for providing external dependencies to the application (for example databases, mail servers, ftp servers, ....) you might find a big problem and it is that probably <b>Docker Hos</b>t used is one and when running tests in parallel, all of them are going to try to start a container with same name. So when you start the second test (in parallel) you will get a failure regarding that a conflict container name because of trying to start at the same<b> Docker Host</b> two containers with same name or having same binding port in two containers.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
So arrived at this point you can do two things:</div>
<div style="text-align: justify;">
</div>
<ul>
<li style="text-align: justify;">You can have one <b>Docker Host</b> for each parallel test.</li>
<li style="text-align: justify;">You can reuse the same <b>Docker Host</b> and use <b>Arquillian Cube</b> <i>Star Operator</i>.</li>
</ul>
<br />
<div style="text-align: justify;">
<b>Arquillian Cube</b> is an Arquillian extension that can be used to manager <b>Docker</b> containers in your tests.</div>
<div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
To use <b>Arquillian Cube</b> you need a <b>Docker</b> daemon running on a computer (it can be local or not), but probably it will be at local.</div>
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<div>
<b>Arquillian Cube</b> offers three different ways to define container(s):</div>
<div>
<br /></div>
<div>
<ul>
<li>Defining a <i>docker-compose</i> file.</li>
<li>Defining a Container Object.</li>
<li>Using Container Object DSL.</li>
</ul>
<div>
In this example I am going to show you how to use <i>docker-compose</i> and <i>Container Object DSL</i>.</div>
</div>
<div>
<br /></div>
<div>
<div>
<i>Star operator</i> let’s you indicate to <b>Arquillian Cube</b> that you want to generate cube names randomly and can adapt links as well. In this way when you execute your tests in parallel there will be no conflicts because of names or binding ports.</div>
<div>
<br /></div>
<div>
Let’s see an example:</div>
</div>
<div>
<br /></div>
<div>
<script src="https://gist.github.com/lordofthejars/de03ab760ccd19f8962f68632cd40b91.js"></script>
</div>
<div>
<br /></div>
<div>
You can see in <i>docker-compose.yml</i> file an important change on a typical <i>docker-compose</i> file, and it is that the name ends up with star <b>(*)</b> operator [<b>redis*</b>]. This is how you are instructing <b>Arquillian Cube</b> that this name should be generated dynamically for each execution.</div>
<div>
<br /></div>
<div>
Then there are three tests (here only showed the first one) that all of them looks like the same. Basically it prints to console the binding port to connect to the server.</div>
<div>
<br /></div>
<div>
Finally there is <i>build.gradle </i>file, which executes two tests in parallel. So if you run the tests in Gradle (<i>./gradlew test</i>) you'll see that two tests are executed at the same time and when it finish one of them, the remaining test is executed. Then if you inspect the output you'll see next output:</div>
<div>
<br /></div>
<div>
<script src="https://gist.github.com/lordofthejars/df0e7afb5f4102a9138811a4b85d412a.js"></script>
</div>
<div>
<br /></div>
<div>
So as you can see in the log, container name is not <i>redis</i> nor <i>redis*</i>, but <i>redis</i> followed by a <i>UUID</i>. Also you can see that when the output is printed the binding port is different in each case.<br />
<br />
But if you don't want to use <i>docker-compose</i> approach, you can also define container programmatically by using <i>Container Object DSL</i> which also supports <i>star</i> operator. In this case it the example looks like:<br />
<br />
<script src="https://gist.github.com/lordofthejars/47459a24d6a4a50694989d5b0bac8533.js"></script>
The approach is the same, but using Container Objects (you need Arquillian Cube 1.4.0 to run it with Container Objects).</div>
<div>
<br /></div>
<div>
Notice that thanks of this feature you can run the tests with any degree of parallel execution, since <b>Arquillian Cube</b> takes care of naming or port binding issues. Notice that in case of linking between containers, you still need to use the star operator, and it will be resolved at runtime.<br />
<br />
To read more about star operator just check <a href="http://arquillian.org/arquillian-cube/#_parallel_execution">http://arquillian.org/arquillian-cube/#_parallel_execution</a><br />
<br />
Source code: <a href="https://github.com/lordofthejars/parallel-docker">https://github.com/lordofthejars/parallel-docker</a><br />
<br />
We keep learning,<br />
Alex.<br />
<blockquote class="tr_bq">
I can show you the world, Shining, shimmering, splendid, Tell me, princess, now when did, You last let your heart decide? (A Whole New World - Aladdin)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=sVxUUotm1P4">https://www.youtube.com/watch?v=sVxUUotm1P4</a><br />
<br />
<br /></div>
</div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com2tag:blogger.com,1999:blog-19517292.post-84487635747078638602017-05-12T15:12:00.000+02:002017-05-12T15:12:30.255+02:00Testing Spring Data + Spring Boot applications with Arquillian (Part 2)<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-Zy4cJrsu2BM/WP8PEd31JyI/AAAAAAAABfw/2_bMHwJPqygAMT-RTRA938nyGeGqDgXMgCPcB/s1600/boot%252Barq.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="80" src="https://3.bp.blogspot.com/-Zy4cJrsu2BM/WP8PEd31JyI/AAAAAAAABfw/2_bMHwJPqygAMT-RTRA938nyGeGqDgXMgCPcB/s200/boot%252Barq.png" width="200" /></a></div>
<br />
<div style="text-align: justify;">
In <a href="http://www.lordofthejars.com/2017/04/testing-spring-data-spring-boot.html">previous post</a>, I wrote about how to test<b> Spring Data</b> application using <b>Docker</b> with <b>Arquillian Cube</b>. The test looked like:</div>
<div style="text-align: justify;">
<br /></div>
<script src="https://gist.github.com/lordofthejars/cf23e513d13fae30acff1cb8195ea6a4.js"></script>
<br />
<div style="text-align: justify;">
This test just starts <b>Redis</b> container, then populate data using <i>restTemplate</i> and <i>post </i>method, then execute the logic under test (testing <i>GET HTTP</i> method) and finally stop the <b>Redis</b> container.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
It is good, it works but there are several problems there:</div>
<ul>
<li style="text-align: justify;">The first one is that we are using <i>REST API</i> to prepare data set of the test. The problem here is that the test might fail not because a failure on code under test but because of the preparation of the test (insertion of data).</li>
<li style="text-align: justify;">The second one is that if <i>POST</i> endpoint changes format/location, then you need to remember to change everywhere in the tests where it is used.</li>
<li style="text-align: justify;">The last one is that each test should leave the environment as found before execution, so the test is isolated from all executions. The problem is that to do it in this approach you need to delete the previous elements inserted by <i>POST</i>. This means to add <i>DELETE HTTP</i> method which might not be always implemented in endpoint, or it might be restricted to some concrete users so need to deal with special authentication things.</li>
</ul>
<div>
<div style="text-align: justify;">
To avoid this problem <b>Arquillian Persistence Extension</b> (aka <i>APE</i>) was created. This extensions integrates with <b>DBUnit</b> and <b>Flyway</b> for <b>SQL</b> databases, <b>NoSQLUnit</b> for <b>No SQL</b> databases and <b>Postman collections</b> for <b>REST</b> services so you can populate your backend before testing the real test use case and clean the persistence storage after the test is executed.<br />
<br />
Also population data is stored inside a file, so this means that can be reused in all tests and easily changed in case of any schema update.</div>
<br />
Let's see example of <a href="http://www.lordofthejars.com/2017/04/testing-spring-data-spring-boot.html">Part 1</a> of the post but updating to use <b>APE</b>.<br />
<br />
<script src="https://gist.github.com/lordofthejars/860417259d6f7c49eb9d49afdaa0a797.js"></script>
And the file (<i>pings.json</i>) used for populating <b>Redis</b> instance with data looks like:<br />
<br />
<script src="https://gist.github.com/lordofthejars/8719ca319009015268e22f4e1f3ae5c1.js"></script>
<br />
<div style="text-align: justify;">
Notice that in this test you have replaced the <i>POST</i> calls for something that directly inserts into the storage. In this way you avoid any failure that might occurs in the insertion logic (which is not the part under test). Finally after each test method, <b>Redis</b> instance is cleaned so other tests finds <b>Redis</b> clean and into known state. </div>
<br />
Project can be found at <a href="https://github.com/arquillian-testing-microservices/pingpongbootredis">https://github.com/arquillian-testing-microservices/pingpongbootredis</a><br />
<br />
We keep learning,<br />
Alex<br />
<blockquote class="tr_bq">
Y es que no puedo estar así, Las manecillas del reloj, Son el demonio que me tiene hablando solo (Tocado y Hundido - Melendi)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=1JwAr4ZxdMk">https://www.youtube.com/watch?v=1JwAr4ZxdMk</a><br />
<br />
<br />
<br /></div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com5tag:blogger.com,1999:blog-19517292.post-70997748425788521452017-05-02T18:21:00.001+02:002017-05-02T18:21:25.404+02:00Testing Dockerized SQL Databases<div class="separator" style="clear: both; text-align: center;">
<a href="https://upload.wikimedia.org/wikipedia/commons/thumb/4/49/Ape_skeletons.png/510px-Ape_skeletons.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="172" src="https://upload.wikimedia.org/wikipedia/commons/thumb/4/49/Ape_skeletons.png/510px-Ape_skeletons.png" width="320" /></a></div>
<br />
<div style="text-align: justify;">
One of the big advantages of using <b>Docker</b> for testing is that you don't need to install the required dependencies of code under tests in all machines where you are going to run these tests. This is really helpful for external services such as database servers, mail services, JMS queues, ... Also one of the big advantages of this approach is that the tests are going to use the same version used in production.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
So for persistence tests using <b>Docker</b> is a really good approach to follow. But as usually this approach comes with some drawbacks. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<b>The first one</b> is that obviously you need to have <b>Docker</b> installed in all machines that needs to run the tests, not a big problem but something to take into consideration, as well as <b>Docker</b> inside <b>Docker</b> problem.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<b>The second one</b> is that you need to automate somehow the starting and stopping of the container.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<b>The third one</b> is that <b>Docker</b> containers are ephemeral. This means that when you start the container, in this case a container with a <b>SQL</b> server, then you need to migrate the database schema there.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<b>The fourth one</b>, and this is not only related to Docker, is that you need to maintain test method execution isolated from test to test execution, by providing known data before execution and cleaning data after the execution so other test finds the environment clean.</div>
<div style="text-align: justify;">
<br />
<b>First</b> and <b>second</b> problems are fixed with <b>Arquillian Cube</b> (<a href="http://arquillian.org/arquillian-cube/">http://arquillian.org/arquillian-cube/</a>). It manages lifecycle of containers by starting and stopping them automatically before and after test class execution. Also it detects when you are running into a <i>DinD</i> situation and configures started containers accordantly.<br />
<br />
<b>Arquillian Cube</b> offers three different ways to define container(s).<br />
<br />
<ul>
<li>Defining a docker-compose file.</li>
<li>Defining a Container Object.</li>
<li>Using Container Object DSL.</li>
</ul>
<br />
For this post, Container Object DSL approach is the one used. To define a container to be started before executing tests and stopped after you only need to write next piece of code.<br />
<div>
<br /></div>
<div>
<script src="https://gist.github.com/lordofthejars/5d2cc19349beafcd552a6eccb14b1468.js"></script>
</div>
<div>
<br /></div>
<div>
In this case a JUnit Rule is used to define which image should be used in the test (<i>redis:3.2.</i>6) and add as binding port the Redis port (6379).</div>
<div>
<br /></div>
<div>
The <b>third</b> one can be fixed using <b>Flyway</b>. It is an open-source database migration tool for <i>SQL</i> databases that allows you to automate the creation of database schemas.<br />
<br />
<b>Flyway</b> is useful here since you can start the <b>Docker</b> container and then apply all migrations to the empty database using <b>Flyway</b>.<br />
<br />
The <b>fourth</b> problem can be fixed by using tools like <b>DBUnit</b>. iI puts your database into a known state between test runs by populating database with known data, and cleaning it after the test execution.<br />
<br />
<b>Arquillian</b> integrates with both of these tools (<b>Flyway</b> and <b>DBUnit</b>) among others with its extension called <b>Arquillian Persistence Extension</b> (aka <b>APE</b>),<br />
<br />
An example on how to use <b>APE</b> with <b>DBUnit</b> is shown in next snippet:<br />
<br />
<script src="https://gist.github.com/lordofthejars/26e067e52fe3916a176385b4729d3e75.js"></script>
You can use <i>Arquillian</i> runner as shown in <a href="https://github.com/arquillian/arquillian-extension-persistence/blob/2.0.0/arquillian-ape-sql/standalone/dbunit-ftest/src/test/java/org/arquillian/ape/rdbms/dbunit/PersonTest.java#L19">dbunit-ftest-example</a> or as shown in previous snippet using a JUnit Rule. Choosing one or other depends on your test requirements.<br />
<br />
So how everything fits together in <b>Arquillian</b> so you can boot up a <b>Docker</b> container with a SQL database, such as <i>PostgreSQL</i>, before test class execution, then migrate SQL schema and populate it with data, execute the test method, then clean the whole database so next test method finds a clean database and finally after test class execution, the <b>Docker</b> container is destroyed?<br />
<br />
Let's see it in the next example:<br />
<br />
<script src="https://gist.github.com/lordofthejars/11a1850ce48b9eed394647725938d3df.js"></script>
Test is not so much complicated and it is pretty much self explanatory of what it is doing in each step . You are creating the <b>Docker</b> container using <b>Arquillian Cube</b> <i>DSL</i>, and also you are configuring the populators by just using <b>Arquillian APE</b> <i>DSL</i>.<br />
<br />
So thanks of <b>Arquillian Cube</b> and <b>Arquillian APE</b> you can make your test totally isolated from your runtime, it will be executed always agains the same PostgreSQL database version and each test method execution will be isolated.<br />
<br />
You can see full code at <a href="https://github.com/arquillian/arquillian-extension-persistence/tree/2.0.0/arquillian-ape-sql/standalone/dbunit-flyway-ftest">https://github.com/arquillian/arquillian-extension-persistence/tree/2.0.0/arquillian-ape-sql/standalone/dbunit-flyway-ftest</a><br />
<br />
We keep learning,<br />
Alex<br />
<blockquote class="tr_bq">
Ya no me importa nada, Ni el día ni la hora, Si lo he perdido todo, Me has dejado en las sombras (Súbeme la Radio - Enrique Iglésias)</blockquote>
Music: <a href="https://www.youtube.com/watch?v=9sg-A-eS6Ig">https://www.youtube.com/watch?v=9sg-A-eS6Ig</a><br />
<br /></div>
</div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com3tag:blogger.com,1999:blog-19517292.post-46993590898617781602017-04-26T14:58:00.000+02:002017-04-26T14:58:01.934+02:00Testing Spring Data + Spring Boot applications with Arquillian (Part 1)<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-Zy4cJrsu2BM/WP8PEd31JyI/AAAAAAAABfs/vaJAP7ULoYUXLaadIUoVLqCtWTYZdPZKwCLcB/s1600/boot%252Barq.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="128" src="https://3.bp.blogspot.com/-Zy4cJrsu2BM/WP8PEd31JyI/AAAAAAAABfs/vaJAP7ULoYUXLaadIUoVLqCtWTYZdPZKwCLcB/s320/boot%252Barq.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<b>Spring Data</b>’s mission is to provide a familiar and consistent, Spring-based programming model for data access while still retaining the special traits of the underlying data store. It provides integration with several backend technologies such as JPA, Rest, MongoDB, Neo4J or Redis to cite a few.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
So if you are using <b>Spring (Boot)</b> then <b>Spring Data</b> is the right choice to deal with persistence layer.</div>
<br />
In next example you can see how simple is to use <b>Spring Boot</b> and <b>Spring Data Redis</b>.<br />
<br />
<script src="https://gist.github.com/lordofthejars/4ae6a603210ce54df213d24ce17a221a.js"></script>
<br />
<div style="text-align: justify;">
It is important to notice that by default <b>Spring Data Redis</b> is configured to connect to <i>localhost</i> and port <i>6379</i>, but you can override those values by setting system properties (<i>spring.redis.host</i> and <i>spring.redis.port</i>) or environment variables (<i>SPRING_REDIS_HOST</i> and <i>SPRING_REDIS_PORT</i>).</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
But now it is time to write a test for this piece of code. The main problem you might get is that you need a Redis server installed in all machines that need to execute these tests such as developers machine or Jenkins slaves. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
This is not a problem per se but when you start working on more and more projects you'll need more and more databases installed on the system, and what even can be worst not exactly the same version as required on production. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
To avoid this problem, one possible solution is using <b>Docker</b> and containers. So instead of relaying on having each database installed on the system, you only depends on <b>Docker</b>. Then the test just starts the repository container, in our case Redis, executes the test(s) and finally stops the container.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
And this is where <b>Arquillian</b> (and <b>Arquillian Cube</b>) helps you on automating everything.</div>
<div style="text-align: justify;">
<b>Arquillian Cube</b> is an <b>Arquillian</b> extension that can be used to manager <b>Docker</b> containers from <b>Arquillian.</b></div>
<div style="text-align: justify;">
<b><br /></b></div>
<div style="text-align: justify;">
To use <b>Arquillian Cube</b> you need a <b>Docker</b> daemon running on a computer (it can be local or not), but probably it will be at local.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
</div>
<div style="text-align: justify;">
By default the <b>Docker</b> server uses <i>UNIX</i> sockets for communicating with the <b>Docker</b> client. <b>Arquillian Cube</b> will attempt to detect the operating system it is running on and either set <i>docker-java</i> to use <i>UNIX</i> socket on <i>Linux</i> or to <i>Boot2Docker/Docker-Machine</i> on <i>Windows/Mac</i> as the default <i>URI, </i>so your test is portable across several <b>Docker</b> installations and you don't need to worry about configuring it, <b>Arquillian Cube</b> adapts to what you have installed.</div>
<div style="text-align: justify;">
<b><br /></b></div>
<div style="text-align: justify;">
<b>Arquillian Cube</b> offers three different ways to define container(s).</div>
<div style="text-align: justify;">
</div>
<ul>
<li>Defining a <i>docker-compose</i> file.</li>
<li>Defining a<i> Container Object</i>.</li>
<li>Using <i>Container Object DSL</i>.</li>
</ul>
<br />
<div style="text-align: justify;">
For this post, Container Object DSL approach is the one used. To define a container to be started before executing tests and stopped after you only need to write next piece of code.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<script src="https://gist.github.com/lordofthejars/5d2cc19349beafcd552a6eccb14b1468.js"></script>
</div>
<div style="text-align: justify;">
In this case a JUnit Rule is used to define which image should be used in the test (<i>redis:3.2.6</i>) and add as binding port the Redis port (<i>6379</i>).<br />
<br />
The full test looks like:<br />
<br />
<script src="https://gist.github.com/lordofthejars/cf23e513d13fae30acff1cb8195ea6a4.js"></script>
Notice that it is a simple <b>Spring Boot</b> test using their bits and bobs, but <b>Arquillian Cube</b> JUnit Rule is used in the test to start and stop the Redis image.<br />
<br />
Last important thing to notice is that test contains an implementation of <i>ApplicationContextInitializer</i> so we can configure environment with Docker data (host and binding port of Redis container) so <b>Spring Data Redis</b> can connect to correct instance.<br />
<br />
Last but not least build.gradle file defines required dependencies, which looks like:<br />
<br />
<script src="https://gist.github.com/lordofthejars/084bcb9fa7a40260f8ae250c3b8d8cff.js"></script>
</div>
<div style="text-align: justify;">
<br />
You can read more about <b>Arquillian Cube</b> at <a href="http://arquillian.org/arquillian-cube/">http://arquillian.org/arquillian-cube/</a></div>
<div style="text-align: justify;">
<br />
We keep learning,<br />
Alex<br />
<br />
<blockquote class="tr_bq">
Hercules and his gifts, Spiderman's control, And Batman with his fists, And clearly I don't see myself upon that list (Something just like this - The Chainsmokers & Coldplay)</blockquote>
<br />
Music: <a href="https://www.youtube.com/watch?v=FM7MFYoylVs">https://www.youtube.com/watch?v=FM7MFYoylVs</a><br />
<br /></div>
Anonymoushttp://www.blogger.com/profile/03336355786853564350noreply@blogger.com3