Is there a way in scrapy to keep track of how a url is reached?
Ex. when url https://test.com/resource_1/resource_2/resource_3/target is downloaded, I need to get a history of how scrapy got there, along with the redirects.
[https://domain1.com/,
.
.
https://test.com/resource_1/resource_2/,
https://test.com/resource_1/resource_2/resource_3/]
Related
I have a Rails application which is suppose to upload images from a Dropbox URL pointed to a folder. The folder contains the images. Application is suppose to upload all images present in folder.
The URL is somewhat like this
https://www.dropbox.com/sh/17fsm6bsnac1g4q/AADJ7B2L0OIrkSrc7YcG-OO9a?dl=0
I can see the images but how can I get the list of all images URL. I have tried parsing the URL by appending dl=1 which downloads the images.
URI.parse('https://www.dropbox.com/sh/17fsm6bsnac1g4q/AADJ7B2L0OIrkSrc7YcG-OO9a?dl=1).
How can I get the URL of all images. If I can not get URL of images then how can i download all images and them upload them.
If you are okay with just downloading everything, you can make a GET request to the dl=1 version of the link you have. This URL parameter is documented here. This will give you a zipped version of the folder which you can then unzip and use as necessary.
Dropbox doesn't offer a way to get links for each of the files in the linked folder, but you can use the Dropbox API to list the files and then download them individually. You can use the /2/files/list_folder endpoint and pass the shared link in as the shared_link parameter. That will give you a list of the items in the folder.
You can then use /2/sharing/get_shared_link_file to download any desired file(s), by passing in the shared link as the url parameter, and the relative path for the file as the path parameter.
I couldnt find similar topic (maybe exists)...
Whenever my browser try to load
example.com/abc.js, instead I want to load this file actually:
mysite.com/abc.js
(p.s. I DONT OWN THOSE SITES, I WANT ONLY TO TRICK MY BROWSER TO LOAD DIFFERENTLY ONLY ONE SPECIFIC FILE.)
with htaccess redirect
Redirect http://come_from/abc.js http://send_to.com/abc.js
I am using swagger-ui and have different servers/interfaces/. So for each of them I store a bookmark - but the URL used in the swagger-ui is never ever saved within the bookmark. I always have to type in the address again.
Is it possible to save the URL used in the swagger-ui in a browser bookmark (for example as a special parameter)?
If your swagger-ui is running here:
file:///Users/me/work/swagger-ui/dist/index.html
And your server is running here:
http://localhost:8080/api-docs
Then you can bookmark the swagger-ui for your server like so (a simple GET with a param named "url"):
file:///Users/me/work/swagger-ui/dist/index.html?url=http://localhost:8080/api-docs
Does anyone know if Brackets Shell uses any URI except the file://...index.html format? I want to point a web service back to the shell and need to provide a valid URI with the URL. The problem is that I noticed that the URL for the index file uses basic system paths and this can change per system or user configuration.
Any way around this little problem?
Edit:
I am trying to access the Instagram API through brackets shell. In order to gain an access token to the OAuth method they use I need to redirect the user to the Instagram Login page to log in then grant access and then Instagram will redirect them back to me based on the call back or redirect url provided from me. If it was as simple as providing just the URL when the call was made I would be fine however to get a client key (which is also needed to access the api) I need to provide the correct call back URL once the key is made and since the shell seems to have no custom file path it's almost impossible to predict the file path based on each users setup on their own pc's. I hope this made sense :)
I searched the source of the brackets-shell, but couldn’t find the startup path. A quick workaround would be adding a meta-refresh to your index.html which redirects to the corresponding URL of your web service.
For example:
<meta http-equiv="refresh" content="0; url=http://example.com">
If you don’t want to select your index.html each time you start your custom shell, you should place your index.html at
Mac: Brackets.app/Contents/dev/src/index.html or Brackets.app/Contents/www/index.html
Win: dev/src/index.html or www/index.html (these folders must be in the same folder as Brackets.exe)
Linux: dev/src/index.html or www/index.html (these folders must be in the same folder as the Brackets executable)
You can get the current location through javascript using
document.location.href
this will return a string like
"file:///C:/Program%20Files%20(x86)/YourApp/www/index.html"
which you might try sending to Instagram. I'm not sure if they will accept file URI's though.
I am a complete beginner in Flash & Actionscript.
My pet project is this: To provide a www.imageshack.com like service where people could upload single images and later anyone can view it using the generated url.
So far I have gotten to upload an image using Flash and store it in a directory.
http://pixels.guygar.com/
You can check the uploaded image at:
http://pixels.guygar.com/warehouse/
The issue being, I was under the impression when the PHP file is called to store the image in the folder /warehouse the browser would automatically navigate to:
http://pixels.guygar.com/upload.php
Where I can process the image i.e. generate a unique file name and provide the user with a unique URL to later access the resource.
What is happening is the image gets uploaded by the PHP script but the browser page still stays the same page even when providing a new url in the PHP script.
So the question is how do I go about so that a new URL (image resource linked) is passed back to the flash so that onComplete is called I can navigate to image that was just loaded? Or other ways of doing such?
I welcome your perspectives on this issue and thank you for your guidance.
i would store all values that you need later on in a session on the server (don't forget to pass the sessionID to the upload-script via GET).
at the end of the PHP script you just return "ok" (or "ko" if sth went wrong) to flash and then (in the callback/listener) call/load a second PHP-script that's doing the rest ... and returns you an URL to a thumbnail or whatever you want to do.
hope this points you in the right direction ...
http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net/FileReference.html?filter_flash=cs5&filter_flashplayer=10.2&filter_air=2.6#event:uploadCompleteData
Shows how data can be returned to flash after an upload.