OpenLiberty Docker image with *all* features enabled - docker

The OpenLiberty Docker Images tagged as full contain a server.xml that only enables the javaee-8.0 feature. Those tagged as microProfile3 only enable microProfile-3.0.
I want both... even better: I'd like to have just all features enabled while I'm developing; I'll optimize for performance when I need it, i.e. maybe not at all.
Is there an easier way than to build another image with both features enabled?

It isn't possible to enable all features at once in Liberty because many of the features intentionally conflict with one another. For example, you can't load two different versions of the same feature at the same time (e.g. servlet-3.1 and servlet-4.0)
You can pretty concisely enable all of the latest JavaEE and MicroProfile features at once by doing this:
<server>
<featureManager>
<feature>javaee-8.0</feature>
<feature>microProfile-3.2</feature>
</featureManager>
</server>
Doing this will give quite a lot of capabilities (more than a single app typically needs). The features not included in these two umbrella features are pretty specialized, such as JCache session persistence (sessionCache-1.0) or event logging (eventLogging-1.0).

You can think of the tags as indicators for what features are included in the image more so than what's enabled by default. In other words, 'full' has all the features available and can be enabled without the need for install, whereas 'microProfile3' only has the microProfile-3 features installed. Note that some packages, like javaee8, have more than just the single feature included as it also provides other features that users may need to use alongside that single feature (though only that one feature is enabled by default). You can see the breakdown of features to package here
Andy's answer explains why you can't enable all the features at once (conflicts). Regarding whether there's an easy way to build with both features enabled, I'd recommend starting with 'full' and updating the Dockerfile to COPY the server.xml with both features (plus any other ones you'd like) to /config. Like you alluded to in your question, this is fine for development, but you would not want to do it for production as it would included a lot of extra features that you're not using. For production, you'd want to use the opposite approach and start with the smallest image (perhaps kernel) and add only the features that your application/server needs, ensuring a fit-for-purpose runtime.

Related

Reduce DeepLearning4j dependency size of exported jar

In my application, I would like to use Deeplearning4j. Deeplearning4j has over 120mb of dependencies, which is a lot considering my own code is only 0.5mb.
Is it possible to reduce the dependencies required? Would loading an already-trained network allow me to ship my application with a smaller file size?
There are many ways to reduce the size of your jar depending on what your use case is. We cover this more recently in our docs, but I'll summarize some things to try here:
DL4j is heavily based on javacpp. You can add -Djavacpp.platform=$YOUR_PLATFORM (linux-x86_64, windows-x86_64,..) to your build to reduce the number of native dependencies in there.
If you are using deeplearning4j-core, that includes a lot of extra dependencies you may not need. In this case, you may only need deeplearning4j-nn for the configuration. The same goes for if you are using only samediff, you do not need the dl4j apis. I don't know enough about your use case to confirm what you do and don't need though.
If you are deploying on an embedded platform, we also have the ability to reduce the number of supported operations and data types now as well. This feature is mainly for advanced users right now (involves building from source) but if you think that could also be applicable despite the first 2, please do confirm and I can try to clarify that a bit.

Is there a mechanism for preserving Processor state between calls

Is there a mechanism for preserving/saving Processor state between calls?
In particular I want a reliable mechanism to know when my process last ran, even if the processor, or even NiFi itself has been restarted.
(Please don't give answers such as hBase or the file system. I am looking for something provided by NiFi, or that can be built with services provided by NiFi)
There is currently no out of the box functionality that automatically captures the listed information unilaterally throughout the application for all processors.
There are mechanisms that provide the capability of accomplishing this type of functionality in components via ControllerServices (think of these as components for cross-cutting concerns or aspects) like the DistributedMapCacheServer/Client or DistributedSetCacheServer/Client.
There are processors that make use of these controller services in manner analogous to your desired feature such as DetectDuplicate or ListHDFS.
This is where things stand currently. There is work under way for the next release (0.5.0) that brings more framework functionality to accomplish such tasks and its work is outlined in our State Management Feature Proposal.
If none of these items quite fits your desired functionality or you have some other ideas, I encourage you to share them with the community either via our mailing lists if you want to hash out your ideas and/or JIRA.

How to perform image processing using the "12 factor app" way in Rails

I'm having a hard time figuring out the best way to do image uploads in my Rails project while still adhering the the '12 factor app' standards.
Basically, I would like to have users upload their own profile images. I know I can use Paperclip to accomplish that, but I still need to use something like ImageMagick to resize the images and prepare the pictures to be used in my application.
According to 12 Factor app's Section 2, Dependencies (http://12factor.net/dependencies):
A twelve-factor app never relies on implicit existence of system-wide packages.
Twelve-factor apps also do not rely on the implicit existence of any system tools. Examples include shelling out to ImageMagick or curl.
It says you shouldn't use ImageMagick locally. That makes sense, but how would you do you image processing then?
Thanks in advance for any advice you can give!
I think the key word there is implicit.
Make dependencies explicit, and account for behaviors if some required component you don't ship is not present. There are ton's of environment management tools that let you stand up a new environment configured in the 'proper' way.
I tend to think of ImageMagick as a backing service, akin to the database your application uses. I don't believe 12-factor is telling you that you have to also ship the RDBMS in order to be totally self-contained.
Perhaps some configuration values where if specified, then the ImageMagick libraries are used, or degrade gracefully until libraries are loaded and configured.
Yeah, it's a sticky wicket.

CMS, or pre-baked solutions for community file sharing

I want to create a community around a current iPhone app I've built. It will allow registered users to upload and download small configuration or settings files, which are used in my app to customize functionality. These files are serialized plists (binary files around 500 bytes), but can be converted to a JSON or XML format if necessary.
I do not need an HTML front-end; I plan for it to be accessed only via my app. Files do not need to be private or secure. I do not plan to store or ask for any user private data--just a login and password.
I'm looking for tips that might get me close to my goals with the least amount of effort - I want to focus on the core functionality of the app, and have this as a stable feature that I can add to in the future if it is useful. I would of course prefer FOSS, but a commercial solution is not out of the question. Things like file sharing sites with apis, login ideas, and so on.
So, what software solutions are out there that I may not be aware of? I know that Drupal has modules to allow user logins. Is there something that would work not as a web app, but as a service only? Dropbox has file sharing and an API, but I'm not sure I could use it the way I'm intending.
In short, I could code this, but would prefer a pre-baked solution that would deal with things I may not have thought of. I am sure there must be something out there which I can use.
More Details, and what I plan on the service offering:
Registration of users via the iPhone, and all that entails (will code the UI myself--I just want an API to connect to)
Viewing of these files quickly and efficiently (the files were built with performance in mind, and this is a free app, so I would like to keep server costs down)
Uploading their own files, with a few integrity checks
Rating the files
Gathering statistics on usage (which files were downloaded most often), etc., to provide a way for the files to be ranked by rating, popularity, etc.
Optional - submitting revised versions of the files (a tree).
Optional but preferred - statistics on users (no. files uploaded, perhaps rewards system for sharing)
I'm just not up to date with current technologies and open source solutions. I have experience in SQL, relational database design, and have built backends in Java, so a custom solution is not out of the question. However, it's been a while, I'm not a security expert, and would prefer to not reinvent the wheel for what is a fairly simple project, so an off-the-shelf solution would be preferred.
Check out www.parse.com!
It is absolutely brilliant for stuff like this.
You may want to look at source versioning systems like SVN or distributed systems like Mercurial or GIT. Both would be much better if the data were serialized to a text format, like JSON or XML as you mentioned.
Registration would need to be done by you of course
Viewing of files (including changes, of course) is quick and efficient. The interface can be done in a number of ways, even simulating command-line.
Uploading files will of course work, and changes made will be stored as diffs. Integrity checks can be done, for example, by Mercurial plugins
Rating the files probably can't be done directly unless you wanted an awkward hack involving parsing change entries or writing a plugin.
Submitting revised versions of files would work as that is the raison d'ĂȘtre of versioning systems.
Some statistics are made available in VCSs.
This is honestly a bit of a strange use for version control systems and not altogether elegant, but sometimes that's what innovation is about.
I suggest TikiWiki .
Pros:
Out-of-the-box all you need to build a community. (See reference below for list of features)
It's FOSS
It has 200 active developers - so it really has a lot of momentum.
Cons:
So many out-of-the-box features that it suffers from feature bloat. Configuration and initial set-up may be complicated.
Not really oriented to mobile platforms.

Tools to manage semantic webs

I've seen a lot frameworks to create a semantic web (or rather the model below it). What tools are there to create a small semantic web or repository on the desktop, for example for personal information management.
Please include information how easy these are to use for a casual user, (in contrast to someone who has worked in this area for years). So I'd like to hear which tools can create a repository without a lot of types and where you can type the nodes later, as you learn about your problem domain.
For personal semantic information management on the desktop there is NEPOMUK. There are two versions, one embedded in kde4, this lets you tag, rate and comment things such as files, folders, pictures, mp3s, etc. on the desktop across all applications.
Another version is written in Java and is OS independent, this is more of a research prototype. It has more features, but is overall less stable.
For KDE-Nepomuk see http://nepomuk.kde.org/
For Java-Nepomuk see http://dev.nepomuk.semanticdesktop.org/ and http://dev.nepomuk.semanticdesktop.org/download/ for downloads (the DFKI version is better)
Extensive list of semantic web tools
Also check out Protege
If you need to create a small model, then I suggest that you use topbraid. I have used for creating much larger models and I know people who have used to create humongous models. It comes packaged with a set of reasoners and provides ability to plug-in custom reasoner and in case if you decide to make your model larger, you can even integrate Topbraid with a triple store like Allegrograph.
And since its based on eclipse, to get started with it is relatively easier.
For developers who are spoiled working in more matured programming languages like Java (IDEA ? anyone), topbraid is the closest tool to an actual IDE.
Chandler is a "a notebook you can organize, back up and share!" It seems to be pretty simple to use.
OS: Windows, Mac, Linux

Resources