I am curious if it is possible to automatically measure whether a test suite is flaky from the Circle CI interface. I would measure flaky as - fail and pass with a re-trigger. Is this possible to easily do?
Not at the moment, as far as I'm aware. I've done an extensive research on build insights in general, which included flaky tests analysis and monitoring, and finally decided to build my own tool. The good news is that last I checked, they seem to be focusing on creating better insights tools in addition to what they currently have. They'll tell you all about it if you reach out to them.
In the interim, you have a few options:
Ask them how far away are they from supporting for your idea of what a flaky is (I'm hoping this point gets outdated shortly as they work on it)
Consume their data through their decent enough API, and build your own tool in the interim and crunch the numbers yourself (this is what I ended up doing and it isn't too bad)
For example: generally speaking, a flaky for my team is a test that failed more than a few times over a large timespan. Their API gives you whether a build failed or not, which test failed, when and how. This gave me enough to work with and figure out whether I consider that spec failure as flaky or not. I'd assume yours is sort of similar, with maybe the only difference being whether it was re-triggered (unsure if they provide that info specifically, but you could refer to the workflow, commit and build ID to figure that out; e.g. if the build ID of a new run is the same).
With that beind said, the "how easy is it?" part of your question is something I can't really say for certain. It was a relatively easy learning curve to go through their APIs, get familiar with it, run a couple of requests, look at the data, massage it, store in the DB, then build a web interface around it. But I'm not sure how much familiarity and experience the people building the tool on your end have.
I've been a java developer for most of my career, and developed and maintained some pretty large code bases.
now that I've been developing in rails for a while I've left my IDE behind and I want so start looking at the actual rails source. I've mainly been using the github interface which is actually quite good but I wondered if there were tips and techniques from more seasoned ruby devs?
What I'm missing mainly is my trusted eclipse features like "find usage" or "goto implementaton". Perhaps a paradigm shift is in order... please enlighten me :P
Since you're an Eclipse user I'd recommend Rubymine. It gives you a lot of the same features and has good integration with rails. It has good support for jumping to implementation/usage, although this is a harder problem in ruby than java. When Rubymine can't determine which implementation is the correct one, you're usually given a list of all matching methods.
I personally use Vi and haven't had trouble navigating rails projects, usually it's a matter of convention and familiarity that just comes with working on the same project for enough time. I default to using grep or find for usages etc.
The problem is that things like "goto implementation" and its ilk may require deep knowledge of the runtime. Some methods may not exist until after an initialization or mixin process. Duck typing means that anything that responds to a message is a potential type candidate.
I'm reasonably happy with IntelliJ (Rubymine), but you won't find the same level of navigational support you get from a language like Java, where most everything is known up front. It's simply less navigable than Java code, IMO.
The things that can be known, like explicitly-defined methods, "obvious" calls, and so on are navigable, and IMO right now IntelliJ does it the best, but I haven't used an Eclipse Ruby environment for some time, so the situation may be different now--but I'm often surprised at how well it does do, considering the circumstances.
That said, I flip back-and-forth between IntelliJ, TextMate, Sublime Text 2, and Emacs. TextMate is nearly useless to me because of its single-pane design. For quick stuff or small projects, it's adequate.
I have an idea for a large web application, which in my mind would really be a number of reasonably independent application "components" (?), e.g. forum, blog etc.
It seems to me that it would be nice to open source some of the individual components for others, or myself in the future to use.
Now clearly, I could write a basic forum, make the code available, then copy it across into my large app and extend/integrate it. However, this doesn't feel very DRY. In an ideal world I could maintain the code in one place and drop it into my larger application.
Rails seems to include engines, which sound like solve my problem but it would appear that what I want to do is a bad idea!
So should I..
a) Just build one large, coupled app.
b) Create a rails forum app (for example), release it, then merge/copy it into my main app
c) Use engines
d) Other...
Cheers,
Adam
That warning is just the standard Uncle Ben Corollary. Paraphrased from "With great power, also comes great responsibility." to "Just because you can, doesn't mean you should."
However, just because there are a number of uses for engines that are bad ideas, that doesn't mean it is in your case. Blogs and forums are two reasonably independent components that have already been produced as plugins using Engines. You might not even need to make them.
The way I see it, if you're planning on reusing a functional component that you can feasibly segregate from the rest of the app, then that's a good enough reason to make it a plugin, whether or not it relies on engines.
My experience is that these plugins grow from existing application code I've written. It's much easier to conceive, write, and test them as part of an application than on their own. Engines being miniature apps, aren't that much harder to build and test.
In short. Only you can decide which path is best, because you are most familiar with the goals of your app. Before you start have a look at what others have done with engines and plugins. The components you want to build may already exist.
To answer the question, I would probably start by building a single app, and extract out the bits that could be useful in other apps and abstract them into [engine] plugins as they reach maturity.
What you might be looking for Git Submodules. I don't know a ton about them, but you can simply have a git repo within a git repo. And you can update all your inner git repos pretty easily. Here's an article talking about it.
What language, framework, and hosting considerations should one make before starting development of a scalable web application?
The most important consideration is not to over-engineer to the point that it gets in the way of building and launching something. Analysis paralysis is the single biggest inhibitor to productivity, progress and results.
Yes, do some planning. Pick a framework. Perfection in a framework will be impossible to find because it doesn't exist, partially because you don't know what you need until you build it anyways. Chances are, if you pick something, it will be better than picking nothing.
Yes, try to pick flexible, inter-operable tools for where you see yourself going.
Yes, look for a good built-in feature set where you see yourself going in the next 6-18 Months. Trying to look beyond that is not really realistic anyways as most projects change so much anyways going towards the first release.
So, pick what you're comfortable with or what is familiar. Don't follow the crowd, do what gets you the best results, quickest, and often. Understand that you might have to change in the future. So, whatever you build now, try to use unit testing so you can re-factor if ever needed.
If what you're building is going to be super successful, it will be a great problem to have, and an easy one to work on once it's making money as you'll be able to get other talent to help you.
Share what you end up picking and why for your situation -- it helps the us learn from you too!
Don't necessarily marry yourself to one language or framework. It may be that some parts of your site work better with different languages and frameworks than others. For example, all of 37signals' sites are based on Ruby on Rails, but they recently wrote a blog post about how the underlying technology of one is actually written in Erlang now because it's much easier to do concurrency that way.
Obviously there's a level of complexity where things turn into a mishmash, but using the right tool for the job — even if that means different tools for different jobs — can simplify things.
Firstly on language, it largely doesn't matter. PHP, Java and .Net being probably the biggest three are all proven in the sense that they run some of the largest sites on the Web so don't listen to anyone who tells you one is more suited than any of the others.
Some might also put Ruby and Django/Python in this list. I have nothing against them but I'm not aware of any big (say top 50) sites using either.
Hosting considerations depend on how low you want to start but basically the order is:
Shared;
Virtual Private Server;
Dedicated.
Scalability will largely be about your application's design than any language, framework or provider. Efficient database schema, efficient delivery and use of Javascript/CSS and in-memory caching are all issues common to any language or framework.
Language - I'd recommend something with good frameworks and good testing libraries like Perl or Java.
Framework - it depends on what do you plan to do. If you start with a hosting that does not allow FastCGI, it is best to avoid such frameworks like Catalyst or Rails. That's why I love CGI::Application (primarily Perl, but ported to other languages too) - it can run as CGI, FastCGI or mod_perl. For development it can be run from it's own web server.
Hosting - nothing is better than you own server. It can be your own server, leased server or virtual server. But you can start with cheapest hosting and when you need more, you should be able to afford it.
It depends.
Start by looking at your requirements (Functional or user defined) (Non Functional - aspects that describe your desired system link text)
Next I would clarify what it means to have a scalable web application. Define it as test cases that can be clearly tested (must support X page views / second with response time < Y seconds).
Once I had those pieces in place I would look at what type of skills my development team can support (for the intial project and on going maintenance). Then find some case studies of applications out in the wild that use similar language or framework. If someone else has made a specific language / framework scale then chances are good that you can too.
Finally go out and look for some hosting providers that support your chosen language, framework and requirements.
For example, I'm running into developers and architects who are scared to death of Rails apps, but love the idea of writing new Grails apps.
From what I've seen, there is a LOT of resource overhead that goes into using the JVM to support languages such as Groovy, JRuby and Jython instead of straight Ruby or Python.
Ruby and Python can both be interpreted on just about any OS, so I don't see any "write once run anywhere" advantage... why bring the hulking JVM along with you?
Java is a much, much more mature platform, with a lot of existing class libraries that could be "dropped in" and used, than, say, Ruby or Python (or even Perl, for that matter). So for people who like using existing code, rather than writing everything themselves, Java is a huge win.
For example, recently I've been looking for something like JAXB for Python or Ruby. In the end, I ended up using JRuby just because I haven't found any mature, widely-used XML-binding libraries.
The huge advantage of writing code (in any language) for the JVM is that it's usually very easy to tap into the enormous wealth of mature Java libraries out there, if necessary.
And I don't know where you got this idea of a "hulking" JVM with a huge resource overhead. The JIT tends to produce code that is quite fast, and the core JVM is anything but huge by today's standards. It does tend to have a huge memory footprint when running, but that's because modern machines have a lot of RAM and the GC works best when it has a lot of RAM to play with. If desired, the GC can be fine-tuned to hell and back to be more conservative.
As someone else put it: "The best thing about Groovy is that I don't have to use Java. The second best thing about Groovy is that I can use Java".
An assumption that seems to be built into the question is that new projects are greenfield projects. Many organizations have made a huge investment in Java over the last decade+ and require any new project to work within the existing (internal) code ecosystem. As pointed out, there's a huge bonus in all the publicly available Java libraries (whether free/OSS or commercial), but the need to work with existing code and even as a component within an existing system is at least as important (if not more so) to large organizations.
A lot also comes down to the maturity and capability of the platform, which is to say the JVM and everything that comes with it (the entire Java ecosystem). A few examples off the top of my head:
You can plug a remote debugger into a running JVM and get all kinds of information about a running application that is simply impossible with Python, Ruby, etc. Going a step further, there's JMX, a standard way to write code so that objects can be monitored and even tweaked in a live application. Take a look at JConsole and see if you don't drool just a little (despite the ugliness of the interface).
Going even further in this direction, there's OSGi, a standard for writing highly modular code that can be deployed, started, stopped, and even upgraded in a live application. With OSGi you break a large application into many smaller "bundles" which can then be maintained (deployed, started/stopped, upgraded) separately. This is a really big deal in large applications, or any applications that need to remain running at all times.
The platform has very good support for asynchronous, reliable messaging. You get JMS as a baseline, and many excellent and powerful libraries built on it for doing complicated things with very little code (cf. Apache Camel, ServiceMix, Mule, and many others). This is another feature that's extremely valuable in larger applications or those which must run within a larger code universe.
The JVM has real (OS-level) threading, while Python et al. are very limited in this regard (notoriously so). (That being said, shared state concurrency -- threading -- is the wrong approach; cf. Erlang, Alice, Mozart/Oz, etc.)
There are numerous JVM choices beyond the standard Sun implementations, like JRockit, IBM's JVM, etc. This is a developing area with other languages -- Python has Jython, Iron Python, even PyPy and Stackless; Ruby has JRuby, Rubinius, and others -- but as good as these are they can't match the maturity found in the various JVM offerings.
All that being said, I really don't like Java the language and avoid it as much as possible. These days with all the excellent alternative languages for the JVM I don't have to. Groovy gets my vote for its accessibility and tight integration with the platform (and even the language), and because of Grails, which I sometimes like to call "Rails for grownups". I like other JVM languages better, particularly Clojure and Scala, but these aren't as accessible to the average programmer. Scala is popping up a lot lately, though, especially thanks to its high profile use at Twitter, so there's hope for interesting and truly excellent languages making it in larger environments. But that's another topic.
why bring to hulking JVM along with you?
JVM isn't bloated, nor is it slow. on the contrary, it's a lean, fast, deeply optimized VM. Unfortunately, it's optimized for static OOP languages.
Still, good compilers targeting JVM do create good performing programs. I don't know about JRuby; but Jython's goal is to be all-around faster than regular C Python, and they're getting close (it's already faster at several important use cases).
Remember that a good JIT (like those for JVM) can apply some optimizations unavailable on static C compilers, getting faster code from them isn't a pipe dream. Of course, a VM optimized for your language should be faster than a 'not-really-generic' VM like JVM; but there's the maturity issue: JVM has a lot of work done there, while JITs for Ruby and Python aren't anywhere near.
Unfortunately, there doesn't seem to be any better generic bytecode VM. Microsoft's CLI suffers from similar limitations as JVM (ironPython is much slower and heavier than JPython). The best candidate seems to be LLVM. Does anybody know why isn't there more dynamic languages over LLVM? I've seen a couple of Scheme compilers, but seem to have several problems.
Groovy is NOT an interpreted language, it is a dynamic language. The groovy compiler produces JVM bytecode that runs inside the JVM just like any other java class. In this sense, groovy is just like java, simply adding syntax to the java language that is meaningful only to developers and not to the JVM.
Developer productivity, ease and flexibility of syntax make groovy attractive to the java ecosystem - ruby or python would be as attractive if they resulted in java bytecode (see jython).
Java developers are not really scared of ruby; as a matter of fact many quickly embrace groovy or jython both close to ruby and python. What they don't care about is leaving such an amazing platform (java) for a less performant, less scalable even less used language such as ruby (for all its merits).
The big knock on RoR is that it isn't scalable and hard to deploy. By using the Java platform, you can leverage your existing infrastructure.
grails war
Produces a war file that is easily deployed on Glassfish, Jboss, etc.
Ruby and Python can both be
interpreted on just about any OS, so I
don't see any "write once run
anywhere" advantage... why bring the
hulking JVM along with you?
Mostly because you want to take advantage of the HUGE existing ecosystem of Java libraries, APIs and products, which dwarfs anything available for Ruby or Python, especially in the enterprise domain.
Also, keep in mind that JRuby and Jython are faster in a lot of benchmarks than the regular (C implementations) of the languages, especially Ruby (even Ruby 1.9).
Having multiple languages targeting the same virtual machine has a lot of benefits, such as leveraging a common infrastructure, code reuse, shared APIs, the ability to use whatever language is conceptually best for you, or for a specific problem domain, etc.
The same things happens in the .NET space, with multiple languages targeting the CLR. The Parrot (vaporware) VM project also aims to the same thing, and it's a stated goal of the LLVM project too.
The reason is Hotspot.
It is an engineering tour de force.
the other reason not many mentioned is existing infrastructure related to jvm - if you already have a server running java stuff, why not use it instead of bringing in yet another platform (like rails)?
I've encountered this and also been baffled by it, and here's my theory.
Enterprise software is full of Java programmers. Like programmers of all stripes, many Java programmers are convinced that their language is the fastest, the most flexible and the easiest to use--they're not too familiar with other languages but are convinced that those who practice them must be savages and barbarians, because any enlightened person would, of course, use Java.
These people have built vast, complicated Java infrastructures: rube-goldberg machines of frameworks and auto-generated code full of byzantine inheritance structures and very, very large XML files.
So, when someone comes along and says "Hey! Let's use a C interpreted language! It's fast and has neat libraries and is much quicker for scripting and prototyping!" The Java guy is firstly like "I have to run a make file to configure this? QUEL HORREUR!" Then the reality of having to deploy and host this on servers that are running dated OSes and dated versions of Tomcat and nothing else starts to set in.
"Hey, I know! There's a java version of this interpreted language! It may break down in the fast lane on the bridge in rush-hour, and it sometimes catches on fire, but I can get Tomcat to run it. I don't have to dirty my hands with learning non-java stuff, and I can shoehorn it into the existing infrastructure! Win!"
So, is this the "right" reason for choosing a java implementation of a scripting language? Probably not. Depends on your definition of "right". But, I suspect that it's the reason they're chosen more often than snobs like me would like to believe.