Getting reverse dependencies of a module in Ivy - ant

Question

We are using Ivy for storing our binaries and managing dependencies.
With the purpose of managing the impact of changes in modules, we would need to gather this information from the repository:
Given a module name, organization, branch and revision, obtain all modules that are directly or transitively dependent on that module (with branch and revision). Particularly interesting are the impacted "top-level" (application) modules.
Is there any tool suitable for this task? Otherwise, what would you suggest to solve it?
I've tried the repreport task without much success, as it doesn't seem appropriate to browse the dependencies in reverse.

Answer 1

We have Jenkins, and have a bunch of jars we build that other applications depend upon. We use a Maven repository for storing these jars. In Jenkins, a developer can take a particular jar build, and promote that jar into our Maven repo.
The problem is that this jar breaks some of the applications that use that jar. Therefore, I want to be able to build those projects whenever a jar was promoted to our Maven repo. This list gives me the names of the Jenkins projects that depend upon a particular jar.
Now, our projects were built with ant pre-ivy, so I created several macros that helped developers use Ivy. We have replaced the <jar> task with the <jar.macro/> task. It is like the <jar> task except I take the ivy.xml, convert it into a Maven pom.xml and embed it in the jar. I take a look at the build.xml for these <jar.macro> tasks so I know the names of the jars it builds. You'll probably have to munge this to fit your own.
The following Perl script goes through our Subversion projects, looks for the ones with an ivy.xml. Goes through the build.xml, sees what jars are built, associates them with the project, and then goes through the ivy.xml to see what projects it depends upon.
You're welcome to use it.
#! /usr/bin/env perl
#
use warnings;
use strict;
use autodie;
use feature qw(say);
use XML::Simple;
use Data::Dumper;
use File::Find;
use Pod::Usage;
use constant {
SVN_REPO => 'http://svn/rsvp',
IVY_XML_FILE => 'ivy.xml',
SVN => '/usr/local/bin/svn',
};
#
# These are projects we don't want to include, but
# they have 'ivy.xml' in them anyway
#
use constant BAD_PROJECTS => qw( # These are projects with an ivy.xml, but we don't want
...
);
my %bad_projects = map { $_ => 1 } BAD_PROJECTS;
#
# Find the branch or use trunk (First parameter used
#
my $branch = shift; # This is the branch to search on
if ( not defined $branch ) {
$branch = "trunk";
}
my $branch_url;
if ( $branch eq "trunk" ) {
$branch_url = $branch;
}
else {
$branch_url = "branches/$branch";
}
#
# Use "svn ls" to find all the projects that have an ivy.xml
#
open my $project_fh, "-|", "#{[SVN]} ls #{[SVN_REPO]}/$branch_url";
my %ivy_projects;
say "FINDING IVY PROJECTS";
while ( my $svn_project_name = <$project_fh> ) {
chomp $svn_project_name;
$svn_project_name =~ s|/$||; # Remove the trailing slash
next if exists $bad_projects{$svn_project_name};
#
# See if an ivy.xml file exists in this project via "svn ls"
#
my $svn_ivy_project_url = SVN_REPO . "/$branch_url/$svn_project_name";
my $ivy_file = "$svn_ivy_project_url/ivy.xml";
my $error = system qq( #{[SVN]} ls $ivy_file > /dev/null 2>&1 );
next if $error; # No ivy.xml
say " " x 4 . "Ivy Project: $svn_ivy_project_url";
#
# Ivy project exists. Create a new "project" object to store all the info
#
my $project = Local::Project->new($svn_ivy_project_url);
my $ivy_xml = qx( #{[SVN]} cat $svn_ivy_project_url/ivy.xml );
$project->Ivy_xml( $ivy_xml );
my $build_xml = qx( #{[SVN]} cat $svn_ivy_project_url/build.xml );
$project->Build_xml( $build_xml );
$ivy_projects{ $svn_project_name } = $project;
}
#
# Go through build.xml files and look for all jar.macro tasks. Go through
# these and map the Ivy Artifact Name to the project that builds it.
# The Ivy Artifact Name could be from the ivy.xml file. However, if
# the paramter pom.artifact.name exists, the Ivy Artifact Name will be that.
#
my %jars_to_project_name;
for my $svn_project ( sort keys %ivy_projects ) {
my $project = $ivy_projects{$svn_project};
my $url = $project->Url;
my $build_ref = $project->Build_ref;
my $ivy_ref = $project->Ivy_ref;
my $build_xml_project_name = $build_ref->{name};
say qq(Parsing build.xml of "$svn_project");
#
# Go through all targets looking for jar.macro tasks
#
for my $target ( keys %{ $build_ref->{target} } ) {
next unless $build_ref->{target}->{$target}->{"jar.macro"};
#
# Contains a Jar Macro Task: This could be an array reference?
#
my #jar_macros;
my $jar_macro_task = $build_ref->{target}->{$target}->{"jar.macro"};
if ( ref $jar_macro_task eq "ARRAY" ) {
#jar_macros = #{ $jar_macro_task };
} else {
#jar_macros = ( $jar_macro_task );
}
for my $jar_macro ( #jar_macros ) {
#
# If there is no "pom.artifact.name" in the jar.macro
# task, we need to use the name of the module in the
# ivy.xml file. If pom.artifact.name does exist, we will
# use that. We also need to find out if the name contains
# "${ant.project.name}". If it does, we need to replace that
# name with the name of the build.xml project entity name.
#
my $ivy_jar_name;
if ( not exists $jar_macro->{"pom.artifact.name"} ) {
$ivy_jar_name = $ivy_ref->{info}->{module};
}
else { # Name of jar is in the jar.macro task
$ivy_jar_name = $jar_macro->{"pom.artifact.name"};
my $ant_project_name = $build_ref->{name};
$ivy_jar_name =~ s/\${ant\.project\.name}/$build_xml_project_name/;
}
$jars_to_project_name{$ivy_jar_name} = $svn_project
}
}
}
#
# At this point, we now have all of the information in the ivy.xml file
# and the mapping of artifact name to the project they're in in Subversion.
#
# Now, we need to go through the ivy.xml files, find all com.travelclick
# artifact dependencies, and map them back to the SVN projects.
#
#
# A Hashes of Arrays. This will be keyed by BASE JAR svn project. The array
# will be a list of all the other projects that depend upon that BASE JAR svn
# project.
#
my %project_dependencies;
say "MAPPING IVY.XML back to the dependent projects.";
for my $project ( sort keys %ivy_projects ) {
say "On $project";
my $ivy_ref = $ivy_projects{$project}->Ivy_ref;
my $dependencies_ref = $ivy_ref->{dependencies}->{dependency};
for my $dependency ( sort keys %{ $dependencies_ref } ) {
next unless exists $dependencies_ref->{$dependency}->{org};
next unless $dependencies_ref->{$dependency}->{org} eq 'com.travelclick';
#
# This is a TravelClick Dependency. Map this back to the SVN Project
# which produced this jar.
#
# We now have the SVN project that contained the dependent jar and the
# svn project that mentions that jar in the ivy.xml project.
#
my $svn_project = $jars_to_project_name{$dependency};
next if not $svn_project;
if ( not exists $project_dependencies{$svn_project} ) {
$project_dependencies{$svn_project} = {};
}
$project_dependencies{$svn_project}->{$project} = 1;
}
}
for my $project ( sort { lc $a cmp lc $b } keys %project_dependencies ) {
printf "%-30.30s", "$project - ";
say join ( "-$branch,", sort { lc $a cmp lc $b } keys %{ $project_dependencies{$project} } ) . "-$branch";
}
package Local::Project;
use XML::Simple;
sub new {
my $class = shift;
my $project_url = shift;
my $self = {};
bless $self, $class;
$self->Url($project_url);
return $self;
}
sub Url {
my $self = shift;
my $url = shift;
if ( defined $url ) {
$self->{URL} = $url;
}
return $self->{URL};
}
sub Ivy_xml {
my $self = shift;
my $ivy_xml = shift;
if ( defined $ivy_xml ) {
$self->{IVY_XML} = $ivy_xml;
$self->Ivy_ref($ivy_xml); #Generate the ref structure while you're at it.
}
return $self->{IVY_XML};
}
sub Build_xml {
my $self = shift;
my $build_xml = shift;
if ( defined $build_xml ) {
$self->{BUILD_XML} = $build_xml;
$self->Build_ref($build_xml); #Generate the ref structure while you're at it.
}
return $self->{BUILD_XML};
}
sub Ivy_ref {
my $self = shift;
my $ivy_xml = shift;
if ( defined $ivy_xml ) {
$self->{IVY_REF} = XMLin("$ivy_xml");
}
return $self->{IVY_REF};
}
sub Build_ref {
my $self = shift;
my $build_xml = shift;
if ( defined $build_xml ) {
$self->{BUILD_REF} = XMLin("$build_xml");
}
return $self->{BUILD_REF};
}
=pod
=head1 NAME
find_dependencies.pl
=head1 SYNOPSIS
$ find_depdendencies.pl [ <branch> ]
Where:
=over 4
=item *
<branch> - Name of the branch. If not given, it is assumed to be trunk.
=back
=head1 DESCRIPTION
This program goes through the Subversion repository looking for
projects that have an C<ivy.xml> file in them. If this file is found,
the project is parsed to see what jars produced by TravelClick it is
dependent upon, and what jars it produces.
Once all of the Subversion projects are parsed. The jars are listed
along with their dependent projects.
The purpose of this program is to build a list of projects that should
be automaticlaly rebuilt when a Jar is promoted to the Maven repository.
=head1 PERL MODULE DEPENDENCIES
This project is dependent upon the following Perl modules that must be
installed before this program can be executed:
=over 4
=item XML::Simple
=back
This project must also be executed on Perl 5.12 or greater.
=head1 BUGS
=over 4
=item *
The list assumes Jenkins projects are named after the Subversion
project with the branch or trunk tacked on to the end.
=back