Errors range from 404s, IOExceptions, or encoding exceptions. They can be buried in the error stack, and occasionally suggest a versioning problem.
How can I prevent or address this class of errors?
The Dataflow service's SDKs and worker take dependencies on common third-party components, which themselves import various dependencies. Version collisions can result in unexpected behavior in the service. If you are using any of these packages in your code, be aware that some libraries are not forward-compatible and you may need to pin to the listed versions that will be in scope during execution. In order to determine whether your JAR has a conflicting version in use, consider inspecting the dependency tree of your project. Consult the list of specifically pinned versions if you suspect a problem here, and also avoid using "latest" for any of these libraries.
I know most projects have minimum versions defined in bower.json for a lot of libraries.
Something I don't understand is that would this be risky that if something updates and have break-changes or bugs, it will affect your application without you knowing it? What is the design thinking behind this?
Thanks!
We have just faced that exact issue on a project I am on, and the solution was to change our bower.json file to target fixed versions.
Specifically, we were targeting angularjs ^1.4.8. In AngularJS v1.6,
$location now uses '!' as the default hash-prefix for hash-bang URLs,
instead of the empty string
For better or for worse (mostly for worse) we had some hard-coded urls in a different application that pointed to this project that broke once bower automatically installed AngularJS 1.6 as part of our automated build process.
The solution was to simply lock down our versions rather than relying on the latest bug fix (i.e prefixing version numbers with ~) or minor build (i.e. prefixing version numbers with ^).
I think the reason that package managers like bower and npm default to dynamic versions is that they rely on semantic versioning, and in theory you should only encounter breaking changes when the major version number changes. Semver uses a major, minor, bugfix pattern. When the bugfix value is incremented it indicates that one or more backwards-compatible bugs have been resolved. When the minor version is incremented it indicates that new backwards-compatible functionality has been added. When the major version is incremented it indicates that new, breaking changes have been introduced.
The problem with this is that firstly, it relies on the package developer to respect the semver rules when they make changes to their packages, and secondly, even when semver is respected it can still lead to problems (as in the example I provided above).
I'm trying to start a purescript project that uses the purescript signal and drawing libraries. When I try to bower install with both of them as dependencies, bower's interactive prompt asks to resolve several dependency versions. I admittedly pick the versions somewhat arbitrarily since most of the issues are several layers of dependencies deep. Then when I pulp build a basic hello world piece of code that doesn't even use the dependencies, pulp gives around 40 errors depending on which versions I selected.
How do I resolve these dependency versions correctly? Sifting through forty errors and all of the dependencies for both projects doesn't seem feasible. Is there some non-arbitrary way of picking the versions that will resolve the conflicts? From the choices it gave me, it didn't seem like there was any clear resolution. Or are their dependencies too conflicting and you just can't use them together?
This may actually be more of an question about bower, but it may also be something about purescript and pulp that I am just unaware of. They both seem like libraries that should be able to work together, and I can't imagine that other people are having too much difficulty using two or more libraries in a single project.
Purescript 0.10 has breaking changes. If you have the latest compiler, it's likely that it's not compatible with some libraries. From what I see, signal lib is updated but the drawing is not. You can update the lib and use your own version. Or, send a PR, Phil and purescript community in general, is really fast on responding to pull requests.
Another temporary solution is to use the old compiler (0.9) and wait a couple months till everything is back to normal.
also see 0.10 migration guide.
We are using this openAM 9.5 RC1 branch source in our project. https://github.com/svn2github/openam.git
In order to fix some bugs, we have to modify existing openAM's amserver library. For this, we have downloaded the source code from above location and tried to compile it offline using ant (as stated in README). But we are not able to compile it anyhow (even after making necessary changes, adding dependencies etc.)
Is there any way to construct required library (amserver.jar) from this source code ?
The OpenAM 9.5.x and 10.0.x versions are rather difficult to build, but from 11.0.0 the build process should be much more simpler since the project has been migrated over to Maven build system.
In any case, the version you are using (Snapshot 9.5.1 RC1) is very much outdated and most likely has several critical issues (not to mention the security issues).
I would strongly advise against putting effort into backporting fixes to that ancient version. Instead you should realize that you are running a more than 4 years old version of a security component, and upgrade your system to a more recent version as soon as possible.
Apache Maven is a very popular build and dependency management tool in the Java open source ecosphere. I did some tests to find out if it can handle compiled Free Pascal / Delphi units and found it easy to implement. So it would be possible to
release open source libraries precompiled for Free Pascal (or Delphi) in a public Maven repository
include metadata in this repository which contains dependency information
use Maven on the command line to download the open source library from the public repository, and automatically resolve all dependencies
local repositories, working as proxies, could be used to cache frequently used binaries
automatic checksum generation and verification (provided by Maven) would reduce the risk of downloading corrupted binaries
source code and even documentation files could be provided with the binaries
binaries can be provided with or without debug information
continuous integration servers like Hudson, TeamCity or CruiseControl can be used to build projects whenever changes have been submitted to the source control system and notify developers about build errors
This way of dependency management could be very beneficial for open source projects which use many third party libraries with complex dependencies. It would avoid typical conflicts caused by using wrong versions.
For the developer, the workflow for editing and building a project would be reduced to a minimum:
checkout the project source from internal version control system
edit source file(s)
run mvn package to automatically download all required third party libraries (precompiled units) if they are not yet in the workstation's local repository
compile and run
The only additional file for Apache Maven which is required in the project folder is the POM.XML file containing the project information.
Edit: while Maven is usable for some of the required tasks, implementing a solution like Maven in native Free Pascal would have some advantages: no Java SDK required, support for all development platforms where Free Pascal is available, maintenance and plugin development in Pascal.
Usage of a Maven-like tool would not be helpful for open source projects only - commercial projects could access and use the artifacts in public Maven repositories in the same way as well.
Maven features are listed at http://maven.apache.org/maven-features.html
Update:
one use case could be the build of Lazarus, where Maven would download all required libraries and invoke the compiler with the necessary build path arguments. Changes in the dependencies on lower levels would be propagated automatically up to the parent build.
Possible benefits:
less time needed to set up a new work
station, no manual installation of
third party libraries required
less errors caused by wrong library
versions, detection of version
conflicts (for example if two
libraries depend on different
versions of a third library)
artifacts which are created inhouse
can be added to the local maven
repository and shared between
developers and project, central
storage of all artifacts with
metadata
builds are reproducible, just by
using the same source and project
metadata file (pom.xml)
can reduce development time and
increase project stability
Update #2: FPMake
the FPMake build system for Free Pascal seems to be a tool with much potential, in many details it is quite similar to Maven:
FPMake is a pascal based build system developed for and distributed with FPC
FPMake standardizes the building by defining some limits like standard directories
the command fppkg <packagename> will look in a database for the package, extract it, and then compile fpmake.pp and run it
it has standard build targets (clean, build, install, ...)
it can create a 'manifest' file suitable for import into a repository (like mvn deploy or mvn install), the manifest is an XML file which looks very similar to a pom.xml in Maven:
FPMake manifest file:
<packages>
<package name="my-package">
<version major="0" minor="7" micro="6" build="1"/>
<filename>my-package-0.7.6-1.zip</filename>
<author>my name</author>
<license>GPL</license>
<homepageurl>http://www.freepascal.org/</homepageurl>
<email>myname#freepascal.org</email>
<description>this is the package description</description>
<dependencies>
<dependency>
<package packagename="rtl"/>
</dependency>
</dependencies>
</package>
</packages>
Freepascal has been working on a package system of its own in a cross between apt-get and freebsd ports style. (download source/build/install automatically), called fppkg.
However work has stalled. People investing time are the bottleneck, not people wanting to choose tools.
As far as Maven goes, I don't like auxilary tools that need installation of huge external runtimes. It might be fine for a big major app (like Open Office), but not for an util.
I also prefer a tool that is designed to the FPC reality and workflow.
Documentation tools, build tools, download systems, testsuite systems are already all there, it just need a person that dedicates a lot of time into it to make it happen.
Some typical problems when introducing a new technology in a project as FPC, and why it has a tendency to make its own tools:
need to train 20+ committers in parttime.
The only COMMON programming language you can assume is Free Pascal. Even Delphi inner workings can't be taken for granted to be known (many committers came directly to FPC or even still via TP or a Mac Pascal)
Obviously that makes something with plugins in a different language annoying.
Bash script is a close second. (g)make third, but already a magnitude less.
All servers are *nix-like (FreeBSD, OS X, Linux), but not all run Apache. (e.g. my FreeBSD mirror runs XSHTTPD)
somebody most knowledgable must be dedicated maintainer for a long time. Fix problems, update/ do migrations etc. Perferably more than one for obvious reasons.
a major pain are Linux distributions (and FreeBSD to a lesser degree), most maintainers of *nix packages are not capable of more than "./configure;make;make install", and must be spoonfed with a near buildable repository and auxilary files.
In-distribution packaging of FPC/Lazarus has always been important, and is still increasing
All distributions have their own special rules about metadata, depedancies, and how sources must be published. Particularly Debian/Ubuntu is very bureaucratic and slow.
Most don't like third party auto-installers on top of their systems (since that bypasses their dependancy control)
This all leads to the effective practice that own tools in Pascal with minimal scripting work best. Some tools used:
Gmake is mainly used to parameterise the build process on a per directory level, a successor, fpcmake (not really a make derivative despite the name) has begun, but the migration hasn't completed.
Latex and a latex to html conversion (tex4ht, but debian uses hevea) are used in the documentation building (the non library documentation)
The community site (netscape community server which uses TCL scripting, a heavy complex application server) has been a trouble ever since it started, but specially lately since the maintainer became less active.
Mantis has been a problem (specially the email module would crash or lame the server due to the volume), but it has been whipped into shape during successive updates and hard work of several lazarus devels. Currently it is a decent workhorse.
lazarus.freepascal.org PHPBB forum OTOH is relatively painless since a lot of younger people know how to deal with it.
The same goes for subversions (though the more advanced scale needs some adjusting, not everybody is deep into the ins and outs of mergetracking)
If somebody was really serious about Maven, I usually would ask him:
to CRITICIALLY investigate the use for the project. In a very concrete way, with schedule and time estimates. Birds-eye level "everything's possible" overviews are essentialy worthless.
Give some thought on future change of used technologies. Every technology is eventually replaced, even the in-house ones, in 18 year+ projects. A new technology must not make migrations of other infrastructural components hard or involved. The new technology to end all new technologies doesn't exist.
Make a migration plan. Migration is often underrated and underestimated.
And in the end, there is always the 1000000 Euro question, who will do the daily maintenance?
Keep in mind that in a company you just kick the person responsible for the application server. But in an informal environment this is way harder, specially long term, since people's lives, occupations and time spent on the project vary.
Sounds like an interesting plan, but the Delphi community (and FPC even more so, I'd imagine!) values libraries as source far more than precompiled libraries. The general consensus is that anyone who uses a binary-only library is a fool, for two reasons: You can't fix any bugs you find in it, and compiler changes will break compatibility.