since AWS announced that they have now finally MacOS machines in their portfolio, and they are advertise it that was setup for customers to use it for their iOS CI / CD, I want to try that as well. Since I'm very new to the AWS ecosystem, I'm not really aware of what AWS provides overall which I could use for that.
I saw that they provide the macOS in a EC2 and also as a on demand service.
Status Quo:
I host my Repository in GitLab
I have a gitlab CI where I run the iOS pipeline through a curl in azure pipelines. (you pay for a agent per month and my experience with their stability is very bad)
What I want to achieve:
I host my Repository in GitLab
...
...
Run the iOS Pipeline on an AWS EC2 macOS instance on demand.
I already had a look into a lot of how to's but I always end up that I was not able to choose a macOS instance.
You can use AWS EC2 Mac but it is a bad choice. It requires a minimum allocation period of 24 hours at $1.083 per hour. With this price you have plenty of choices.
MacStadium.com - so far most stable cloud mac I have used. You have to setup runner by yourself, price begin from $59/mo.
GitHub Actions Mac runner - Ease to use with all software pre-installed. Work best with GitHub repo, can also work with Gitlab with a little twist. Free for 200 minutes/mo.
macOS Runners on GitLab.com - Work best with Gitlab but still in close beta. Price not decide yet. Use MacStadium under the hook.
bitrise / buildkite / buddybuild etc. all very good if you don't mind they take care of everything for you.
Related
I have read four tutorials about getting started with Jenkins, and whilst they say it is possible to run Jenkins on the same computer on develops on they also all recommend installing it on a separate one, most commonly a Mac Mini. However: I only own a MacBook Pro; am short on cash; and am only person contributing to my iOS projects currently (I want to learn Jenkins for future client work). So it would be better for me for now to use my MacBook for both purposes.
Whilst I appreciate this is a matter of opinion somewhat, I am wondering what the reason is for the recommendation of separation, and whether I might be able to run Jenkins on the MacBook for now?
Thank you for reading.
The reason it is advised to have a master server and a number of slave server is only valid in company (or big team) environment. It is that build job can be CPU and memory intensive and often many developer starts jobs on the server. In cases like that one machine (being the master and slave server ot once) will be slow. Not only the jobs will take longer to finish, but even the web interface may become unresponsive.
For learning the basic configuration steps one machine is totally enough and you can even run your builds with your Jenkins instance.
I'm not entirely sure what the reason for that is in those tutorials, however, I can suggest an easy way to get started with Jenkins for free (That's how I usually run jenkins for personal use). You can create a free account with one of the Cloud providers like AWS, GCP or Azure and have your jenkins running there. For example, in AWS you can have a 1-year free trial account where you can spin up some free servers. There are many tutorials online, like this one, which will show you step by step of how to get started with Jenkins on AWS. Here are some high-level steps:
Create a free account in AWS (or any other cloud provider)
Spin up an EC2 instance - it can be any linux version or windows, whatever you are more comfortable with
SSH or RDP to the instance and install jenkins - there are exact installation steps for any flavor of your OS out there
Once the installation is complete, you will be able to access jenkins on your browser - in case of AWS, it would be the public ip of the server and default port 8080
For CI purposes I have a need to set up a cluster of build slaves capable of building iOS apps. For now I'm relying on a single MacMini -with the aim to deploy several more in the future- and I'd like to virtualize several slaves on top of it. Some of these virtual slaves will build the iOS app, others will be smaller Linux slaves for miscellaneous purposes.
I'm completely new to Docker, so my main question is whether it's possible to dockerize Xcode 9.2 and/or MacOS in order to virtualize my iOS build slaves. I've seen very little literature out there on whether this can be achieved and I've found some images in hub.docker.com but they're not documented and don't appear to be very popular.
I'm going through a Docker tutorial right now and eventually will be attempting this -and if I'm successful I'll be answering my own question here for the benefit of others- but given the lack of information I have doubts on whether it is even possible or where I should even start.
Any tips or pointers on this would be greatly appreciated.
Or if anyone knows for fact that this is not possible and can explain why, that would also save me a lot of time.
OS X does not use the Linux kernel, so it cannot run in a Docker container
XCode is not open-sourced and does not have a Linux installer, so it cannot be used in a Linux Docker image.
It seems like your best bet is to build a Packer template using something like packer-macos osx-vm-templates and integrate that into your pipeline.
Look at Docker-OSX which runs macOS with Xcode support inside Docker.
You can connect to that macOS via SSH or VNC. It might be possible to use the same approach in CI/CD.
Related link from readme: "I want to use Docker-OSX for CI/CD-related purposes (sign into Xcode, Transporter)"
I have recently started using jenkins as a CI server. Is there a free jenkins hosting service?
I made a searched on Google, but unfortunately didn't find anything.
Consider a service like travis-ci, which is free for opensource projects. They also offer an enterprise service.
There is also CircleCI, also with an enterprise service.
You can run it locally, especially, it would be easier to run it as a container. Please see this link: https://www.jenkins.io/doc/book/installing/docker/
Due to a hardware issue, I had to change my work station to another Mac for a few weeks.
It took me a couple of hours to setup everything: Android Studio, git, Apache, MySql, etc...
Could I use a docker image to bundle all my development tools ?
(My goal is to have a "backup" of my development environment that I could start running right away on another machine)
Could I use a docker image to bundle all my development tools ?
That means all your dev tools would be Linux tools working in Linux container, on a Linux host.
You would need to provide that Linux host (on your Mac) through a boot2docker Virtual Machine.
But that also mean you could not directly type "git" from a Mac shell, you would need to connect to your VM first in order to launch your 'git' container and run dome docker run --name=git commands.
So no, this doesn't seem to be a good fit for your backup plan on Mac.
Not necessarily. It kinda depends what you are looking for in a development environment.
I do use it for part of my dev env though.
Vagrant + Docker
My personal approach is to rely on Vagrant to fire up a bunch of environments, some of which being full-fledged VMs and others being lightweight containers.
This is a rather controversial approach though, many people would not agree with it, as the tools overlap, both in terms of platform capability and provisioning.
Docker Containers for 3rd Party Services
My personal approach for this is to use Vagrant to fire up a bunch of different VMs, where one is my main dev VM with tools I use for development (IDEs, editors, SCM tools, etc...), and the rest are Docker containers for 3rd party apps that relate to my daily activities (IRC client, database servers like MySQL or MongoDB, etc...).
This fits my cycle decently as these types of tools (like databases) are not something you generally interact with directly through a tty, but something I'd rather connect to with another tool via an API. So I don't need direct access to them, and I do want them to be isolated and easy to jumpstart and dispose of when I jump between projects.
So, docker containers fit part of my idea of a dev environment, but not necessarily all of it.
Just my use case though. Hope it helps.
Shameless plug: Docker Shell
This tool lets you set up a uniform cross platform development environment inside a docker container.
http://dockershell.io/
It is the first time I have ever launched live a website (with Grails web framework under Amazon EC2 platform and Cloud Foundry) and I realized quickly that I am not ready for monitoring and maintening correctly my application in production mode (fortunately the website is accessible to a very limited number of users) .
The issues I have faced so far are:
Cannot change my views. I need to redeploy my application
I have no monitoring. I don't know who is connected, when do they sign in / sign out...
Redploying my application (upload WAR + deploy) takes at least 30 minutes.
I don't know how to restart my Tomcat server without a redeploy through Cloud Foundry !
...
So, my question is very simple:
What tools (including grails plugins) and methods can you recommend me for taking me out from my current blindness?
I am not sure how much this will help, however I use the JavaMelody Grails Plugin(http://www.grails.org/plugin/grails-melody) I use it to see if the site is being used before I pull down the service.
Hope that helps.
I tried Cloud Foundry in it's early days and found it a little rough. Sounds like that's still the case with 30+ minute deploys and the inability to restart your tomcat server. Half of your problems could be solved if you just created your own EC2 instance, installed tomcat and managed your own deployments. That'll let you bounce tomcat through shell access:
sudo /etc/init.d/tomcat6 restart
and redeploy your app
sudo /etc/init.d/tomcat6/stop
cp my.war /to/tomcat/dir
sudo /etc/init.d/tomcat6 start
or else you could do it through the tomcat admin console, but I find it to be flaky.
Regarding monitoring, there are a couple of ways you can do that. The easiest is to add CloudWatch monitoring to your system. That'll give you more insight into the performance of the application.
For more detailed monitoring regarding who's connecting to your app, I'd suggest looking at Google Analytics or Mint. If you need to get beyond that (with per user monitoring), you'll likely have to roll your own logging/tracking for what meets your needs. There are also other paid packages out there along the lines of Google Analytics and Mint that you can integrate with, but what fits your needs best, I can't say.
For actual monitoring of deployed system you can also use Hiperic HQ. It's a monitoring solution from Spring Source, who also are owners of Grails Framework.
It can manage, at your case:
tomcat server
database
linux
network
etc
btw redeploying app with changes is ok. it's a very bad practice to modify running app, on the production server.
I don't know if JMX and JConsole can help, but that might be a good way to see what the status of JMX-enabled POJOs is. Spring makes this easy to do.
Yes, you need to re-deploy your app when you change stuff in it, there's no way around that.
Deploy/re-deploy time has been cut significantly recently if using the grails plugin (btw, what version of the plugin do you use?) In some cases the upload time is as short as 15 seconds. Add 2-3 minutes for Amazon to spin up the instances and the deploy time is still pretty manageable. For re-deploy the instances don't have to be started, so it's even less than that.
To restart Tomcat login into your CloudFoundry account, click on the Deployment details, click on the instance that's running your Tomcat, and there will be a button "Restart service" that will do just that - restart Tomcat service.
You don't have to start your own EC2 instances in order to get a shell access. Copy the public DNS name of the instance from DeploymentDetails, and SSH into it using the private key you entered when registering for Cloudfoundry. Example:
ssh -i /path/to/gsg-keypair.pem root#your_instance_DNS_name