I have read four tutorials about getting started with Jenkins, and whilst they say it is possible to run Jenkins on the same computer on develops on they also all recommend installing it on a separate one, most commonly a Mac Mini. However: I only own a MacBook Pro; am short on cash; and am only person contributing to my iOS projects currently (I want to learn Jenkins for future client work). So it would be better for me for now to use my MacBook for both purposes.
Whilst I appreciate this is a matter of opinion somewhat, I am wondering what the reason is for the recommendation of separation, and whether I might be able to run Jenkins on the MacBook for now?
Thank you for reading.
The reason it is advised to have a master server and a number of slave server is only valid in company (or big team) environment. It is that build job can be CPU and memory intensive and often many developer starts jobs on the server. In cases like that one machine (being the master and slave server ot once) will be slow. Not only the jobs will take longer to finish, but even the web interface may become unresponsive.
For learning the basic configuration steps one machine is totally enough and you can even run your builds with your Jenkins instance.
I'm not entirely sure what the reason for that is in those tutorials, however, I can suggest an easy way to get started with Jenkins for free (That's how I usually run jenkins for personal use). You can create a free account with one of the Cloud providers like AWS, GCP or Azure and have your jenkins running there. For example, in AWS you can have a 1-year free trial account where you can spin up some free servers. There are many tutorials online, like this one, which will show you step by step of how to get started with Jenkins on AWS. Here are some high-level steps:
Create a free account in AWS (or any other cloud provider)
Spin up an EC2 instance - it can be any linux version or windows, whatever you are more comfortable with
SSH or RDP to the instance and install jenkins - there are exact installation steps for any flavor of your OS out there
Once the installation is complete, you will be able to access jenkins on your browser - in case of AWS, it would be the public ip of the server and default port 8080
Related
I am trying to find the best way to achieve the following scenario;
I am currently working on getting a complex enterprise web application that consist on:
DB
BPM Engine
SOA Engine
Reporting Engine
Web Application Server
IDE
The applications is currently running in non-prod and prod environment but each environment is independent (no infra as a code, and deployments go from dev -> ... -> prod).
When a new developer comes in, they can't run the system in their local machine as it involves too many components (will come to this later). So they do development in their local machine and to test, they need to publish and deploy to dev. Test, rinse and repeat.
I am currently working on reverse engineer the whole thing so I can get it working on my local machine provided that I can install and run all the components. I am nearly there after fiddling with a lot of configuration, settings, etc.
This work I would like others to use, so they can also run the project in their local machines. In fact, since we will be migrating soon, I would like to pack the whole thing in a way that I can deploy it anywhere (the app already working and configured) and parametrised somehow whether is DEV, SYS, UAT, PROD. This, according to my understanding is what a docker image would do for you correct? You do all the work and then you create an image out of it? Then you can have this image running in a container and that way, other people can 'reuse' your work?
Is this the correct way of doing it? Any hints / comments would be appreciated
Apologies for my writing.
since AWS announced that they have now finally MacOS machines in their portfolio, and they are advertise it that was setup for customers to use it for their iOS CI / CD, I want to try that as well. Since I'm very new to the AWS ecosystem, I'm not really aware of what AWS provides overall which I could use for that.
I saw that they provide the macOS in a EC2 and also as a on demand service.
Status Quo:
I host my Repository in GitLab
I have a gitlab CI where I run the iOS pipeline through a curl in azure pipelines. (you pay for a agent per month and my experience with their stability is very bad)
What I want to achieve:
I host my Repository in GitLab
...
...
Run the iOS Pipeline on an AWS EC2 macOS instance on demand.
I already had a look into a lot of how to's but I always end up that I was not able to choose a macOS instance.
You can use AWS EC2 Mac but it is a bad choice. It requires a minimum allocation period of 24 hours at $1.083 per hour. With this price you have plenty of choices.
MacStadium.com - so far most stable cloud mac I have used. You have to setup runner by yourself, price begin from $59/mo.
GitHub Actions Mac runner - Ease to use with all software pre-installed. Work best with GitHub repo, can also work with Gitlab with a little twist. Free for 200 minutes/mo.
macOS Runners on GitLab.com - Work best with Gitlab but still in close beta. Price not decide yet. Use MacStadium under the hook.
bitrise / buildkite / buddybuild etc. all very good if you don't mind they take care of everything for you.
Due to a hardware issue, I had to change my work station to another Mac for a few weeks.
It took me a couple of hours to setup everything: Android Studio, git, Apache, MySql, etc...
Could I use a docker image to bundle all my development tools ?
(My goal is to have a "backup" of my development environment that I could start running right away on another machine)
Could I use a docker image to bundle all my development tools ?
That means all your dev tools would be Linux tools working in Linux container, on a Linux host.
You would need to provide that Linux host (on your Mac) through a boot2docker Virtual Machine.
But that also mean you could not directly type "git" from a Mac shell, you would need to connect to your VM first in order to launch your 'git' container and run dome docker run --name=git commands.
So no, this doesn't seem to be a good fit for your backup plan on Mac.
Not necessarily. It kinda depends what you are looking for in a development environment.
I do use it for part of my dev env though.
Vagrant + Docker
My personal approach is to rely on Vagrant to fire up a bunch of environments, some of which being full-fledged VMs and others being lightweight containers.
This is a rather controversial approach though, many people would not agree with it, as the tools overlap, both in terms of platform capability and provisioning.
Docker Containers for 3rd Party Services
My personal approach for this is to use Vagrant to fire up a bunch of different VMs, where one is my main dev VM with tools I use for development (IDEs, editors, SCM tools, etc...), and the rest are Docker containers for 3rd party apps that relate to my daily activities (IRC client, database servers like MySQL or MongoDB, etc...).
This fits my cycle decently as these types of tools (like databases) are not something you generally interact with directly through a tty, but something I'd rather connect to with another tool via an API. So I don't need direct access to them, and I do want them to be isolated and easy to jumpstart and dispose of when I jump between projects.
So, docker containers fit part of my idea of a dev environment, but not necessarily all of it.
Just my use case though. Hope it helps.
Shameless plug: Docker Shell
This tool lets you set up a uniform cross platform development environment inside a docker container.
http://dockershell.io/
It is the first time I have ever launched live a website (with Grails web framework under Amazon EC2 platform and Cloud Foundry) and I realized quickly that I am not ready for monitoring and maintening correctly my application in production mode (fortunately the website is accessible to a very limited number of users) .
The issues I have faced so far are:
Cannot change my views. I need to redeploy my application
I have no monitoring. I don't know who is connected, when do they sign in / sign out...
Redploying my application (upload WAR + deploy) takes at least 30 minutes.
I don't know how to restart my Tomcat server without a redeploy through Cloud Foundry !
...
So, my question is very simple:
What tools (including grails plugins) and methods can you recommend me for taking me out from my current blindness?
I am not sure how much this will help, however I use the JavaMelody Grails Plugin(http://www.grails.org/plugin/grails-melody) I use it to see if the site is being used before I pull down the service.
Hope that helps.
I tried Cloud Foundry in it's early days and found it a little rough. Sounds like that's still the case with 30+ minute deploys and the inability to restart your tomcat server. Half of your problems could be solved if you just created your own EC2 instance, installed tomcat and managed your own deployments. That'll let you bounce tomcat through shell access:
sudo /etc/init.d/tomcat6 restart
and redeploy your app
sudo /etc/init.d/tomcat6/stop
cp my.war /to/tomcat/dir
sudo /etc/init.d/tomcat6 start
or else you could do it through the tomcat admin console, but I find it to be flaky.
Regarding monitoring, there are a couple of ways you can do that. The easiest is to add CloudWatch monitoring to your system. That'll give you more insight into the performance of the application.
For more detailed monitoring regarding who's connecting to your app, I'd suggest looking at Google Analytics or Mint. If you need to get beyond that (with per user monitoring), you'll likely have to roll your own logging/tracking for what meets your needs. There are also other paid packages out there along the lines of Google Analytics and Mint that you can integrate with, but what fits your needs best, I can't say.
For actual monitoring of deployed system you can also use Hiperic HQ. It's a monitoring solution from Spring Source, who also are owners of Grails Framework.
It can manage, at your case:
tomcat server
database
linux
network
etc
btw redeploying app with changes is ok. it's a very bad practice to modify running app, on the production server.
I don't know if JMX and JConsole can help, but that might be a good way to see what the status of JMX-enabled POJOs is. Spring makes this easy to do.
Yes, you need to re-deploy your app when you change stuff in it, there's no way around that.
Deploy/re-deploy time has been cut significantly recently if using the grails plugin (btw, what version of the plugin do you use?) In some cases the upload time is as short as 15 seconds. Add 2-3 minutes for Amazon to spin up the instances and the deploy time is still pretty manageable. For re-deploy the instances don't have to be started, so it's even less than that.
To restart Tomcat login into your CloudFoundry account, click on the Deployment details, click on the instance that's running your Tomcat, and there will be a button "Restart service" that will do just that - restart Tomcat service.
You don't have to start your own EC2 instances in order to get a shell access. Copy the public DNS name of the instance from DeploymentDetails, and SSH into it using the private key you entered when registering for Cloudfoundry. Example:
ssh -i /path/to/gsg-keypair.pem root#your_instance_DNS_name
For my programming tasks I use about 2-3 remote servers to deploy and run my code against different conditions. This cannot be emulated locally as the server configuration requires powerful hardware. Most of time I need to stop service, update binaries, start service, view logs in realtime, download logs. Currently I'm doing this manually and over time this becomes a real pain in the ass, especially because the environment is not ideal in terms of network bandwidth, reliability etc.
I just wonder if someone from server programmers have the similar problems and how do you bear with them. Any special tools/hints/secrets?
Sounds like you could use Scripting with Windows PowerShell to automate a lot of the manual tasks that you are performing now.
Unit testing and mocking helps. Instead of having to run all the big powerful stuff all the time, you can save that for a nightly build & smoke or CI server job (most of the time).
Similarly, using of a distributed CI server, like Hudson or Buildbot helps you script the distribution and testing to all the machines.