Is this possible. Based on the documentation it looks like imread does not support anything but local file paths? If it is possible would anyone be so kind as to provide a code sample?
Cheers.
Here is Documentation,
The following remote services are well supported and tested against the main codebase:
Local or Network File System: file:// - the local file system, default in the absence of any protocol.
Hadoop File System: hdfs:// - Hadoop Distributed File System, for resilient, replicated files within a cluster. This uses PyArrow as the backend.
Amazon S3: s3:// - Amazon S3 remote binary store, often used with Amazon EC2, using the library s3fs.
Google Cloud Storage: gcs:// or gs:// - Google Cloud Storage, typically used with Google Compute resource using gcsfs.
Microsoft Azure Storage: adl://, abfs:// or az:// - Microsoft Azure Storage using adlfs.
HTTP(s): http:// or https:// for reading data directly from HTTP web servers.
Check above given documentation for more information
Related
I am trying to set up mass ingestion in IICS. This will be a streaming ingestion task. The source is REST V2.
According to the documentation, I can't provide an absolute path to the Swagger file for this connection. Instead, I need a hosted URL.
I tried hosting the Swagger file on a server that has the Informatica Cloud secure agent installed. When I create a connection everything works.
But when I try to add a connection for mass ingestion I get following error:
What is interesting, I also tried hosting this file on a VM in Azure, and when I try to access this file from a server using an internet browser it works. I can also see requests on the web server, but when I create mass ingestion and define the source I still get an error and I can't see any requests for the Swagger file on the web server.
What is wrong?
I'm trying to make a request in Go client.AnnotateVideo(ctx, &annotateVideoRequest) to the Google Cloud Video Intelligence API using the package cloud.google.com/go/videointelligence/apiv1.
I noticed that if I'm on a Google VM, i don't need any credentials or environment variable because the API says:
For API packages whose import path is starting with "cloud.google.com/go",
such as cloud.google.com/go/storage in this case, if there are no credentials
provided, the client library will look for credentials in the environment.
But I guess I can't authenticate because I'm running a Docker Container inside the Google VM, and I don't know if I really need a credentials file in that docker container, because I don't know if the library automatically creates a credentials file, or it just check if there is a $GOOGLE_APPLICATION_CREDENTIALS and then use that (But that makes no sense. I'm on a GOOGLE VM, and I'm supposed to have that permission).
The error is:
PermissionDenied: The caller does not have permissions
Some links that might be helpful:
https://pkg.go.dev/cloud.google.com/go/storage
https://cloud.google.com/docs/authentication#environment-service-accounts
https://cloud.google.com/docs/authentication/production#auth-cloud-implicit-go
https://cloud.google.com/video-intelligence/docs/common/auth#adc
Thanks in advance!
I am new to spring cloud data flow. I am trying to build a simple http source and rabbitmq sink stream using SCDF stream app.The stream should be deployed on OSCF (Cloud Foundry). Once deployed, the stream should be able to receive HTTP POST Request and send the request data to RabbitMQ.
So far, I have downloaded Data Flow Server using below link and push to cloud foundry. I am using Shall application from my local.
https://dataflow.spring.io/docs/installation/cloudfoundry/cf-cli/.
I also have HTTP Source and RabbitMQ Sink application which is deployed in CF. RabbitMQ service is also bound to sink application.
My question - how can I create a stream using application deployed in CF? Registering app requires HTTP/File/Maven URI but I am not sure how can an app deployed on CF be registered?
Appreciate your help. Please let me know if more details are needed?
Thanks
If you're using the out-of-the-box apps that we ship, the relevant Maven repo configuration is already set within SCDF, so you can freely already deploy the http app, and SCDF would resolve and pull it from the Spring Maven repository and then deploy that application to CF.
However, if you're building custom apps, you can configure your internal/private Maven repositories in SCDF/Skipper and then register your apps using the coordinates from your internal repo.
If Maven is not a viable solution for you on CF, I have seen customers resolve artifacts from s3 buckets and persistent-volume services in CF.
I have successfully uploaded and deployed my grails application on amazon elastic beanstalk with Tomcat 8 and Java 8 on linux ec2 and web app is up and running. It works well when doing REST API calls to and from RDS database. I have a api to upload file to the server from mobile app and from web app frontend. When running this grails app in localhost its works great for this api and uploads files successfully to user.home/{myapplicationDirectory}/somefile path in my Windows OS. But after running this app in elastic beanstalk and trying to upload image from mobile gives NPE as FileNotFoundException
FileNotFoundException occurred when processing request: [POST] /api/images/add
/usr/share/tomcat8/sdpl/images/260519011919.zip (No such file or directory)
Stacktrace follows:
java.io.FileNotFoundException: /usr/share/tomcat8/sdpl/images/260519011919.zip (No such file or directory)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
I have a service to get application data storage directory with this method
def String getApplicationPath() {
return System.getProperty("user.home") + File.separator + "images" + File.separator;
}
Hi as I don't see your full application I don't want to be too presumptuous but as you're using AWS Beanstalk you should consider local file storage to always be a temporary storage. Your server could be terminate and restarted by beanstalk if it stops responding or fails any health checks.
You have other options available, again I don't know if you considered them and have a good reason for using the local file system so forgive me if that's the case, though if not, you could use S3 for the storage of images, then you don't have to worry about disk space, and the images could automatically be then served via AWS's CDN - Cloudfront, thus also reducing load on your app.
Alternatively, when you really want to store these images in the filesystem, you can look at using EFS, the Elastic File System. Your EBS instance could mount the filesystem on startup so it will be always available whenever your instance(s) start.
I didn't suggest using a standard EBS volume, as you can only ever attach a volume to a single instance, if you used EFS, you don't have to worry about space and it can be mounted to multiple instances so is a little more flexible.
Program to publish data using aws-sns is working in ubuntu 14.04 LTS.But the package for the same program does not work in openwrt environment.I know the problem is with configuration file,we have to set an aws configuration file in linux to ~/.aws/ in the file credentials.Where should i add this credentials if i use opewrt?And can i use amazon web services like sns without this configuration file?