I am trying to get the download statistics of a docker image we host on our own artifactory server in a groovy plugin on the server.
So far it looks like the manifest.json file has the statistics I am looking for.
repositories.getStats(RepoPath) should return a StatsInfo object which has getDownloadCount().
However, no matter what I pass to repositories.getStats() it always returns a null object.
I've tried:
docker-local
docker-local/ai
docker-local/ai/latest
docker-local/ai/latest/manifest.json
But still repositories.getStats() return a null object.
I would expect the following to work:
def stats = repositories.getStats(RepoPathFactory.create('docker-local','ai/latest/manifest.json'))
def dlCount = stats.getDownloadCount()
But it keeps returning the error:
"Cannot invoke method getDownloadCount() on null object"
Calling ?stats on the manifest.json through the REST API works and it returns a json formatted string with the download statistics. Ex:
my-artifactory-server.io/api/storage/docker-local/ai/latest/manifest.json?stats
"uri" : "my-artifactory-server.io/api/storage/docker-local/ai/latest/manifest.json",
"downloadCount" : 10,
"lastDownloaded" : 1593160404880,
"lastDownloadedBy" : "Peter Griffin",
"remoteDownloadCount" : 0,
"remoteLastDownloaded" : 0
What am I doing wrong?
What you tried looks ok, may be you need to enforce the stats object has to be StatsInfo
Below is my tested and working code which prints the download count of manifest.json
import org.artifactory.repo.RepoPathFactory
import org.artifactory.fs.StatsInfo
executions {
getDockerStats() {
def statsInfo = (StatsInfo) repositories.getStats(RepoPathFactory.create("docker-local","hello-world/latest/manifest.json"))
log.info("stats download count : ${statsInfo.getDownloadCount()}")
}
}
and in the logs
[getDockerStats:12 ] [http-nio-8081-exec-5] - stats download count : 2
Related
I've added a Lambda Step as the first step in my Sagemaker Pipeline. It processes some data and creates 2 files as part of the output like so:
from sagemaker.workflow.lambda_step import LambdaStep, Lambda, LambdaOutput, LambdaOutputTypeEnum
# lamb_preprocess = LambdaStep(func_arn="")
output_param_1 = LambdaOutput(output_name="status", output_type=LambdaOutputTypeEnum.Integer)
output_param_2 = LambdaOutput(output_name="file_name_a_c_drop", output_type=LambdaOutputTypeEnum.String)
output_param_3 = LambdaOutput(output_name="file_name_q_c_drop", output_type=LambdaOutputTypeEnum.String)
step_lambda = LambdaStep(
name="ProcessingLambda",
lambda_func=Lambda(
function_arn="arn:aws:lambda:us-east-1:xxxxxxxx:function:xxxxx"
),
inputs={
"input_data": input_data,
"input_file": trigger_file,
"input_bucket": trigger_bucket
},
outputs = [
output_param_1, output_param_2, output_param_3
]
)
In my next step, I want to trigger a Processing Job for which I need to pass in the above Lambda function's outputs as it's inputs. I'm trying to do it like so:
inputs = [
ProcessingInput(source=step_lambda.properties.Outputs["file_name_q_c_drop"], destination="/opt/ml/processing/input"),
ProcessingInput(source=step_lambda.properties.Outputs["file_name_a_c_drop"], destination="/opt/ml/processing/input"),
]
However, when the processing step is trying to get created, I get a validation message saying
Object of type Properties is not JSON serializable
I followed the data dependency docs here: https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_model_building_pipeline.html#lambdastep and tried accessing step_lambda.OutputParameters["file_name_a_c_drop"] too but it errored out saying 'LambdaStep' object has no attribute 'OutputParameters'
How do I properly access the return value of a LambdaStep in a Sagemaker pipeline ?
You can access the output as follows - step_lambda.OutputParameters["output1"]. You don't need to add .properties
To access a LambdaStep output in another step you can do this:
step_lambda.properties.Outputs["file_name_a_c_drop"]
Try this
steplambda.properties.ProcessingOutputConfig.Outputs["file_name_q_c_drop"].S3Output.S3Uri
I'm trying to write a script that creates a Docker image using a Jenkins image as base, that is, the first line of my Dockerfile is...
FROM jenkins/jenkins:2.249.3
However I wanna be smart and write a script that gets the latest Jenkins stable version and sed that into my Dockerfile, like this
Dockerfile:
FROM jenkins/jenkins:JENKINS_LATEST_STABLE_VER
$ export JENKINS_LATEST_STABLE_VER=`some_api_call`
$ sed -i "s/JENKINS_LATEST_STABLE_VER/$JENKINS_LATEST_STABLE_VER/g" Dockerfile
$ docker build -t docker_url/jenkins:$JENKINS_LATEST_STABLE_VER .
$ docker push docker_url/jenkins:$JENKINS_LATEST_STABLE_VER
I'm aware of jenkins/jenkins:lts but I NEED the actual version number, e.g., 2.249.3. What is "some_api_call" ?
I saw this post, and tried this. However it returns Alpine images, but I need CentOS ones. For the life of me I can't figure out the URL for the Centos images? I essentially want to get this list.
// Import the JsonSlurper class to parse Dockerhub API response
import groovy.json.JsonSlurper
// Set the URL we want to read from, it is MySQL from official Library for this example, limited to 20 results only.
// docker_image_tags_url = "https://hub.docker.com/v2/repositories/library/mysql/tags/?page_size=20"
docker_image_tags_url = "https://hub.docker.com/v2/repositories/library/jenkins/tags?page_size=30"
try {
// Set requirements for the HTTP GET request, you can add Content-Type headers and so on...
def http_client = new URL(docker_image_tags_url).openConnection() as HttpURLConnection
http_client.setRequestMethod('GET')
// Run the HTTP request
http_client.connect()
// Prepare a variable where we save parsed JSON as a HashMap, it's good for our use case, as we just need the 'name' of each tag.
def dockerhub_response = [:]
// Check if we got HTTP 200, otherwise exit
if (http_client.responseCode == 200) {
dockerhub_response = new JsonSlurper().parseText(http_client.inputStream.getText('UTF-8'))
} else {
println("HTTP response error")
System.exit(0)
}
// Prepare a List to collect the tag names into
def image_tag_list = []
// Iterate the HashMap of all Tags and grab only their "names" into our List
dockerhub_response.results.each { tag_metadata ->
image_tag_list.add(tag_metadata.name)
}
// The returned value MUST be a Groovy type of List or a related type (inherited from List)
// It is necessary for the Active Choice plugin to display results in a combo-box
return image_tag_list.sort()
} catch (Exception e) {
// handle exceptions like timeout, connection errors, etc.
println(e)
}
I have created a redis lua script to execute a command based on key data type :-
local keyType = redis.call("TYPE", KEYS[1])
if (keyType == "string")
then
return redis.call("GET",KEYS[1])
else
return nil
end
It is returning null every time I am executing it.Can any please help in correcting the script.
The response to redis.call returns as a table that looks like this: {"ok": "string"} (if the type is a string of course)
So in order to properly check, you should change your code to:
local keyType = redis.call("TYPE", KEYS[1]).ok
and the rest of the code will work fine.
The issue is this: the TYPE command is one of the few commands that return a "simple string" or "status" redis reply (see the redis protocol specs for the response types). In the redis lua documentation it's stated that:
Redis status reply -> Lua table with a single ok field containing the status
Which is what happened here.
Using [1] will also work.local keyType = redis.call("TYPE", KEYS[1]) return keyType[1]
I'm running a test with Grails 2.3.8 and an external configuration, yet the value doesn't seem to be coming through. I've tried this in older versions and am not seeing the same error. Did something change or am I missing something that I fat fingered? I am using an absolute path and the file definitely exists because Grails sees the key.
Config.groovy
reliable.string = "This string is working"
grails.config.locations = ["file:/home/howes/Project/project/test-config.groovy"]
/home/howes/Project/project/test-config.groovy
test.externalstring = "this is a test"
To test it I made a controller and just called grailsApplication to pull out the values. The first time I load the page I get a blank map, and when I refresh I see the key but no value. To make sure I am pulling everything correctly I am outputting the test-config.groovy item and one from the basic Config.groovy.
TestContoller.groovy
class TestController {
def grailsApplication
def index() {
println grailsApplication.config.test
println grailsApplication.config.reliable
return [ext:grailsApplication.config.test.externalstring, ext2:grailsApplication.config.reliable.string]
}
}
Console Output (First Load)
[:]
[string:This string is working]
Console Output (Page Refresh)
[externalstring:[:]]
[string:This string is working]
What am I missing here?
I am currently using neo4j 1.8.1 . I am getting NotInTransactionException , when I am querying the neo4j index to get some nodes.
Following is a simple query , which i am executing on neo4j
if (graphDb.index().existsForNodes("NODEINDEX")) {
IndexHits<Node> hits = graphDb.index().forNodes(NODEINDEX).query(query);
}
The following is stacktrace for the exception.
"message" : "Error fetching transaction for current thread",
"exception" : "NotInTransactionException",
"stacktrace" : [ "org.neo4j.kernel.impl.index.IndexConnectionBroker.getCurrentTransaction(IndexConnectionBroker.java:134)", "org.neo4j.kernel.impl.index.IndexConnectionBroker.acquireReadOnlyResourceConnection(IndexConnectionBroker.java:84)", "org.neo4j.index.impl.lucene.LuceneIndex.getReadOnlyConnection(LuceneIndex.java:105)", "org.neo4j.index.impl.lucene.LuceneIndex.query(LuceneIndex.java:245)", "org.neo4j.index.impl.lucene.LuceneIndex.query(LuceneIndex.java:227)", "org.neo4j.index.impl.lucene.LuceneIndex.query(LuceneIndex.java:238)", "com.uprr.netcontrol.starmap.neo4j.plugins.aggregate_node_status.NodeStatusHelper.getGraphNodes(NodeStatusHelper.java:39)",
I found the following in Neo4j api.
private Transaction getCurrentTransaction() throws NotInTransactionException
{
try
{
return transactionManager.getTransaction();
}
catch ( SystemException se )
{
throw new NotInTransactionException(
"Error fetching transaction for current thread", se );
}
}
Do we need to explicitly start a transaction for querying neo4j index?
Any thoughts?
Thanks
Here's a theory: I don't know if this is only an issue with the code pasted here but the check:
if (graphDb.index().existsForNodes("NODEINDEX"))
checks for the index named "NODEINDEX", however the actual query
graphDb.index().forNodes(NODEINDEX).query(query);
checks for the index named whatever is in the constant NODEINDEX. Those two are probably not the same and so it tries to create that index for you and fails due to not being in a transaction.
if there isn't an existing appropriate index, I think it'll create one before returning it; this operation needs to be wrapped in a transaction.