How do I view and audit Dataflow actions like Create/Cancel Job? - google-cloud-dataflow

I'm using Dataflow with a project with many users, and would like to be able to understand when jobs are created and by whom. Similarly, if a job is cancelled, I'd like to confirm the identity of the initiator of this action.

Google Cloud Platform records these events in a per-service audit log, which is viewable both in the Cloud Logging UI as well as the Activity section. To view an event in Activity, navigate to that page and view the relevant entry, as below:
To view an event in the Log Viewer, go to the Logging page and select 'Dataflow' and 'activity', which should filter results to the structured audit log entries. This will contain more information than the Activity entry, and can be exported to Pub/Sub as well:

Related

is it possible to create an OBIEE Analytics on user activity report?

I am a BI administrator and author on my OBIEE (12.2.1.3.0). I need to create an analytics on users' browsing history. And then share it to their manager who is just another normal user himself (not an administrator). So I searched on Google for a solution. but, I didn't find anything other than "Manage Sessions" section in administration. However, that just shows online sessions, but I need the report per user and their last login time and browsing information. Besides, I don't want to give the manager administration privilege to access this section. instead, I want to create an analytics for him. but for analytics I need a relevant "Subject Area". I don't know if there is a table in OBIEE repository where I can query from. or generally, I might be thinking incorrectly and there are other ways to handle this kind of requirement. Any idea?
Actually yes, it is possible. As you may know, OBIEE repository tables, which can be used to show to end users as subject area that this is usgae tracking utility, just track information around sent queries to BI server and not more about users specifically.
In this occasion, you can use a method which called enhanced usage tracking for OBIEE that is presented completely here:
https://www.rittmanmead.com/blog/2016/12/enhanced-usage-tracking-for-obiee-now-available-as-open-source/
This is exactly what you want. All detail information about users activities in OBIEE and enve some redundant ones. You can create a physical table, then add it to repository file and display as subject area to end user with any permission.
Of course, according to your software environment or implementation structure, you are supposed to make some changes in this manner.
I hope this goes well.

Is there a way to disable snapshots of flows?

I am trying to build a flow which regularly gets every JIRA Ticket from my JIRA instance as input data. This works fine however only the first time the flow runs because it saves the retrieved tickets in its snapshot. When I run the flow the second time flowground will not retrieve any JIRA Tickets because they were already retrieved in the first execution. Is there a way to disable this behavior in flowground so that it will retrieve all of the tickets again in the subsequent runs?
just tested the idea to manipulate the snapshot before the connector is executed.
There is a API function for CRUD operations on snapshots. But you cannot edit a snapshot that is owned by a already started flow. You will get an error message in this case:
{"errors":[{"detail":"Can not edit snapshot in started flow","status":409,"code":409,"title":"Conflict",....
So you are able to reset the snapshot before you start the flow only...but this won't really help you...
I do not see another way beside changing the connectors behavior.

Strange Google Dataflow job log entries

Recently my jobs logs in a job details view are full of entries such as:
"Worker configuration: [machine-type] in [zone]."
Jobs themselves seem to work fine, but these entries didn't show up before and I am worried I won't be able to spot meaningful log entries because of it.
Is it something I should be worried about? Do you know how to get rid of them?
Yes, those logs are spammy and are not to be worried about. I have submitted an internal bug to reduce these spammy logs (with this being the first). While it is being fixed, you can familiarize yourself with the Stackdriver Logs Exclusion feature. This allows you to create filters to exclude logs based on a user-defined query.
Here are the steps to exclude specific Datawflow logs:
Navigate to the logs ingestion page
Find the "Dataflow Step" row
Click the right-most button on the same row
Select the "Create Exclusion Filter..." option from the drop-down
Write the query to select which logs you want to exclude
(in your case: resource.type="dataflow_step" "Worker configuration")
Name your filter
Select your percent of logs to exclude (exclude 100% of selected logs is the default)
Click the "Create Exclusion" button
You can view your created exclusion filter in the "Exclusions" tab in the logs ingestion page
And you should not see such logs spamming for newly scheduled jobs now! We've added logic to prevent excessive logging of this kind of message.

Is there a way to display recent work item activity?

Jira has a feature called Activity Streams, which allows for a gadget to be displayed on dashboards to show what recent changes have been made to issues within a particular stream. This could show link changes, attachments added, fields updated, etc. Is there any way to do something like this with a TFS dashboard widget?
I've done some searching, but haven't come across anything similar.
You can install the Activity Feed extension.
With the activity feed, you can see what's happening in your team project within a glance. You can use it on your dashboard by putting a widget on it, or simply open the hub. The Activity Feed gives a summary of all recent changes of work items, commits, pull requests and builds. It tells who changed a task, who logged a bug and who made committed code.

Zendesk report overview API end point

Which API end point can be used to get the Reports Overview data?
I need to have the following data :
New Tickets
Solved Tickets
Open Tickets
Unresolved Tickets
Recently updated tickets
6.Pending Tickets
Also, I would like to pull a leader board of agents by their solved ticket counts and to have an individual report for each agent which shows the above 6 metrics only for the agent.
The view preview count API can be used for your main ones. https://developer.zendesk.com/rest_api/docs/core/views#preview-count
You can do a lot with the other view APIs, and if you don't want to create the view you use previews. Should you need more detail, your fallback is the /tickets.json endpoint to get all the data.

Resources