I have a notebook running as a job in Azure Databricks. The results are shown in a Databricks dashboard. I want the dashboard URL to be sent to the team when the run is finished.
How do I retrieve the URL of the dashboard for the current run ?
I know the job ID, I managed to get the base of the URL with
dbutils.notebook.entry_point.getDbutils().notebook().getContext().browserHostName().toString()
and I found one can get the Run ID with
dbutils.notebook.entry_point.getDbutils().notebook().getContext().currentRunId().toString()
but the URL should contain the "Run", which is different than the "Run ID". Furthermore, the URL doesn’t display a dashboard without some UUID I don’t know how to get. Where can I get these informations ?
Here is my solution :
run_id = None
url = None
try:
run_id = json.loads(dbutils.notebook.entry_point.getDbutils().notebook().getContext().toJson())["tags"]["idInJob"]
url = f"https://XXX.azuredatabricks.net/?o=YYY#job/11/run/{run_id}/dashboard/ZZZ"
except:
pass
I figured XXX, YYY and ZZZ do not change along runs, you will find them by looking at your sample dashboard.
run_id and url will stay None if the notebook is launched in interactive mode.
Related
I have created set of YouTube reporting jobs for a YouTube channel. The jobs were created and run every day as scheduled. However when I go to download the jobs they are all blank.
This is how I authenticate with the API:
def authenticate_from_credentials(API_SERVICE_NAME, API_VERSION):
youtube_client_id = os.environ['youtube_client_id']
youtube_client_secret = os.environ['youtube_client_secret']
youtube_refresh_token = os.environ['youtube_refresh_token']
credentials = client.OAuth2Credentials(
access_token=None,
client_id=youtube_client_id,
client_secret=youtube_client_secret,
refresh_token=youtube_refresh_token,
token_expiry=None,
token_uri='https://oauth2.googleapis.com/token',
user_agent=None,
revoke_uri=None
)
youtube_reporting = build(API_SERVICE_NAME, API_VERSION, credentials=credentials)
return youtube_reporting
This is the method I have been using to create the jobs:
# Call the YouTube Reporting API's jobs.create method to create a job.
def create_reporting_job(youtube_reporting, report_type_id, name):
# Provide keyword arguments that have values as request parameters.
reporting_job = youtube_reporting.jobs().create(
body=dict(
reportTypeId=report_type_id,
name=name
),
).execute()
print ('Reporting job "%s" created for reporting type "%s" at "%s"'
% (reporting_job['name'], reporting_job['reportTypeId'],
reporting_job['createTime']))
I authenticate like this:
youtube_reporting=authenticate_from_credentials('youtubereporting','v1')
And I will create a job like this:
create_reporting_job(youtube_reporting,"channel_combined_a2","Channel Combined a2")
I am not sure what the problem is here. The channel does have content and subscribers so the reports shouldn't be empty. I think there could be an issue with credentials or perhaps the wrong channel is associated with the report since the developer's Google accounts are different than the content owners. But I checked the channels associated with the Oauth credentials I am using and it was the right channel.
Why might my reports be empty and how can I fix this?
I hit the same issue, the problem is that you need to wait several hours for the report to become generated on the backend, at which point re-querying for reports will show results.
There is a subtle mention about this delay on https://developers.google.com/youtube/reporting/v1/reports under Step 3:
The API response to the jobs.create method contains a Job resource,
which specifies the ID that uniquely identifies the job. You can
start retrieving the report within 48 hours of the time that the job
is created, and the first available report will be for the day that
you scheduled the job.
This was quite confusing.
I'm just dipping a toe in the water with Crystal at the moment and, as an exercise, trying to port one of my Python scripts across.
The script in question downloads the 'latest' PDF from a URL which takes the form: "http://somesite.com/download/latest/". When visited that URL automatically redirects to the page for the latest download eg. "http://somesite.com/download/4563/"
I'm having difficulty working out how to implement this in Crystal so that I can grab the actual URL that the redirect ends up on.
In Python I do:
currenturl = urllib.request.urlopen(latesturl)
#above will redirect to URL of format http://somesite.com/download/XXXXX/
#where XXXXX is the current d/load
endurl = currenturl.geturl()
...which gives me the end URL in the "endurl" variable.
But, reading the docs for Crystal's "http/client" I can't see any way to return the actual URL that a redirect ends up on. Is it possible?
Crystal's HTTP::Client currently can't automatically follow redirects.
Please note that you're reading an outdated version of the API docs, the current is at https://crystal-lang.org/api/latest/HTTP/Client.html (I don't think there have been relevant changes between 0.24.1 and 0.26.1 though).
But you can easily access the redirect URL from reading the Location header of an HTTP response:
response = HTTP::Client.get latesturl
endurl = response.headers["Location"]
I'm trying to POST data to a Google Web App to automatically enter some tedious data. Whenever I try I get a response back asking me to log in, the Web App is deployed to be accessible by anyone.
Can I POST data to the form without authentication? If not, what type of authentication is required?
Edit: Quick code sample I threw together:
WebClient client = new WebClient();
var keyValue = new NameValueCollection();
keyValue.Add("agentName", "John Doe");
keyValue.Add("dayOff", "Sunday");
keyValue.Add("startTime", "8:00 AM");
Uri uri = new Uri("mywebapp url");
byte[] response = client.UploadValues(uri, "POST", keyValue);
string result = Encoding.UTF8.GetString(response);
In order to allow anyone to execute your script, even if they are not signed in to a Google account, you need to use the following settings in the "Deploy as web app" dialog:
Project version: if in doubt, select "New" to ensure that you are deploying the latest copy of the script's code.
Execute the app as: select "Me". This ensures that the "even anonymous" option will be available.
Who has access to the app: select "Anyone, even anonymous". (Note that if you select "Anyone", only users that are signed-in to a Google account will be able to execute your script.)
Once you have selected these options, click the "Update" button and copy the script URL. It should look like https://script.google.com/a/macros/<APPS DOMAIN>/s/<SCRIPT KEY>/exec. At this point an unauthenticated GET or POST to the script's URL should be successful.
Be aware that the script's execution will count against your daily Apps Script quota.
I'm working on a SharePoint mobile solution where I'm using the web services exposed in server/_vti_bin/sitedata.asmx, server/_vti_bin/Lists.asmx and server/_vti_bin/copy.asmx.
I'm able to successfully fetch the list of sites, document libraries and files using the services defined in server/_vti_bin/sitedata.asmx.
Now I'm actually trying to upload an image file from Photo Albums available in iOS to SharePoint. For this, I tried using CopyIntoItems web service, where in I'm getting the following error response.
<CopyResult ErrorCode="DestinationInvalid" ErrorMessage="The Copy web service method must be called on the same domain that contains the destination url." DestinationUrl="http://xxxxserveripxxxxxx/Shared Documents/image1.png"/>
But came to know that this service is used only if the file to be uploaded is also from the same source(i.e., from sharepoint).
Is there any other way to upload a file available in iPhone to SharePoint.
Also tired addAttachment service defiend in server/_vti_bin/Lists.asmx but I'm unable to identify the input parameters which requires list name and list Item ID.
I'm trying to upload a file to Shared Documents, so I've List Name value which is the one in curly braces of Shared Documents but now what should be the List Item Id value?
These are the details I've with regard to "Shared Documents" document library.
{
AllowAnonymousAccess = false;
AnonymousViewListItems = false;
BaseTemplate = DocumentLibrary;
BaseType = DocumentLibrary;
DefaultViewUrl = "/Shared Documents/Forms/AllItems.aspx";
Description = "Share a document with the team by adding it to this document library.";
InheritedSecurity = true;
InternalName = "{425F837A-F110-4876-98DE-C92902446935}";
LastModified = "2013-07-26 20:09:58Z";
ReadSecurity = 1;
Title = "Shared Documents";
},
So, I'm using the using InternalName value for listName tag.
What should be the value of listItemID?
Am I going in the right way or is there any other approach to upload a local file from mobile to SharePoint?
Thanks
Sudheer
Are you actually calling a URL or are you using the IP (you x'ed it out and said server IP)? If you don't have Alternate Access Mappings defined for the IP, uploads will fail but the GET requests will generally work ok.
I'm trying to authenticate users coming to my site from a click on an opengraph action in facebook.
I've generate the action successfully with the following code. This
goes out to my timeline successfully. So far, so good.
require 'cgi'
client_id ="CLIENT_ID"
client_secret = "CLIENT_SECRET"
recipe = "http://foodonthetable.shadr.us/recipes/1007-spanish-chicken-and-rice"
access_token = "LONGACCESSTOKEN"
# PUBLISH
result = `curl -F access_token=#{access_token} -F recipe=#{recipe} https://graph.facebook.com/me/foodonthetable_mrh:plan_to_cook`
puts result
When I click on the link in the timeline, I'm taken to the following URL. This is where
I start having issues. Basically, I want to recognize the user coming in
through this link and log them in (or create a new user). I have
authenticated referrals turned on, and I'm receiving the code parameter as
expected.
http://foodonthetable.shadr.us/recipes/1007-spanish-chicken-and-rice?fb_action_ids=384156608318085&fb_action_types=foodonthetable_mrh%3Aplan_to_cook&fb_source=timeline_og&action_object_map=%7B%22384156608318085%22%3A10151004406923956%7D&code=AQAPExNw7MHkxcLTEi5L24iD79pVa-WYxyhBA_bhdWLCM0PCGDuPjh1WqmAyd3_O3_LSjYzPawrinHNP3nv9BCMB_XuTnr8De8xQ2AwXqCaeHUzZUPm2MPyQ_eodOC9-YtjkvXm_PzRX3JG58khalT3AJjVuZvKHn5hWGDSohXLRbHGxW-vOg_Whm3mt_WUkdd0#_=_
However, when I try to get the auth_token using this code, I'm not able to.
I've tried using a couple of different versions of the redirect_uri thinking
that might be the problem (based on other posts) but nothing seems to work.
redirect_uri = "http://foodonthetable.shadr.us/recipes/1007-spanish-chicken-and-rice"
redirect_uri = "http://foodonthetable.shadr.us/facebook_connect/connect"
code = "AQAPExNw7MHkxcLTEi5L24iD79pVa-WYxyhBA_bhdWLCM0PCGDuPjh1WqmAyd3_O3_LSjYzPawrinHNP3nv9BCMB_XuTnr8De8xQ2AwXqCaeHUzZUPm2MPyQ_eodOC9-YtjkvXm_PzRX3JG58khalT3AJjVuZvKHn5hWGDSohXLRbHGxW-vOg_Whm3mt_WUkdd0"
result = `curl -F client_id=#{client_id} -F redirect_uri=#{redirect_uri} -F client_secret=#{client_secret} -F code=#{code} https://graph.facebook.com/oauth/access_token`
puts result
No matter what I do here, I always get the following error:
{"error":{"message":"Error validating verification code.","type":"OAuthException","code":100}}
Any help would be appreciated.
This does fix it.
redirect_uri = request.url.split('&code=').first