Hoping someone has some knowledge on this one. I have a system which allows users to post to facebook or to send a link via email using an ESP called dotMailer. When creating the wall post / email campaign however, both Facebook and dotMailer 'test' the URL embedded in the content to see if it's valid.
I am storing a viewed_on date for the URLs, and as such I want to be able to ignore the HTTP requests by Facebook and dotMailer rather than storing the viewed_on date that they trigger by hitting the URL.
In terms of what I've tried / won't work:
IP Filtering - cannot rely on IP being same each time
Time-based delay - depends on how quickly dotMailer/Facebook processes the requests, so cannot rely on this
Thanks!
I'm a dev with dotMailer - for us, you can rely on the request coming from one of two different places: 94.143.104.0/21, 80.249.97.113, or 80.249.97.114. With Facebook, you can simply check the UserAgent. We use an IE useragent, because a surprising amount of sites behave differently when presented with a non-standard useragent and thus make link checking less reliable.
We've got a great forum, so stop on by if you have any more questions! https://support.dotmailer.com/forums
Related
I am making an api backend that makes use of another api,for example Twitter. (Please note the actual api isn't twitter but I am using them as an example).
Let's say that Twitter has a limit on the number of calls that can be made to their api and above this limit, it starts to charge my credit card. This is why it is very important to me that no one misuses my api.
I want to prevent people from looking at my frontend code and seeing which endpoint it hits, because if a malicious person were to do this, I would very quickly go over the limit and have to pay $$$.
My frontend code uses a get call to mybackend.com/twitter/api
Is it enough to simply add an Access-Control-Allow-Origin header to my backend?
headers['Access-Control-Allow-Origin'] = 'myfrontend.com'
The reason I am asking this is because I noticed that typing mybackend.com/twitter/api directly into the browser worked, which is not what I would expect if I had access-control-allow-origin set to a specific website.
Am I doing something wrong? How do I prevent someone from simply writing a script to hit my backend since it is clear that just typing it into the url of my browser works, despite me having an access-control-allow-origin header.
There are two possible solutions for your problem. You can try to implement a request signature for your API, to know exactly the source of it on your backend. You can take a look on how this works here.
The second option, and for me, a one witch fits your problem better, is to set up a Denial of service approach on your server Load Balancer to prevent multiple requests from a same origin, and so, don't let those kind of malicious requests hit your backend.
I'm making a Rails polling site, which should have results that are very accurate. Users vote using POST links. I've taken pains to make sure users only vote once, and know exactly what they're voting for.
But it occurred to me that third parties with an interest in the results could put up POST links on their own websites, that point to my voting paths. They could skew my results this way, for example by adding a misleading description.
Is there any way of making sure that the requests can only come from my domain? So a link coming from a different domain wouldn't run any of the code in my controller.
There are various things that you'll need to check. First is request.referer, which will tell you the page that referred the link to your site. If it's not your site, you should reject it.
if URI(request.referer).host != my_host
raise ArgumentError.new, "Invalid request from external domain"
end
However, this only protects you from web clients (browsers) that accurately populate the HTTP referer header. And that's assuming that it came from a web page at all. For instance, someone could send a link by email, and an email client is unlikely to provide a referer at all.
In the case of no referer, you can check for that, as well:
if request.referer.blank?
raise ArgumentError.new, "Invalid request from unknown domain"
elsif URI(request.referer).host != my_host
raise ArgumentError.new, "Invalid request from external domain"
end
It's also very easy with simple scripting to spoof the HTTP 'referer', so even if you do get a valid domain, you'll need other checks to ensure that it's a legitimate POST. Script kiddies do this sort of thing all the time, and with a dozen or so lines of Ruby, python, perl, curl, or even VBA, you can simulate interaction by a "real user".
You may want to use something like a request/response key mechanism. In this approach, the link served from your site includes a unique key (that you track) for each visit to the page, and that only someone with that key can vote.
How you identify voters is important, as well. Passive identification techniques are good for non-critical activities, such as serving advertisements or making recommendations. However, this approach regularly fails a measurable percentage of the time when used across the general population. When you also consider the fact that people actually want to corrupt voting activities, it's very easy to suddenly become a target for everyone with a good concept to "beat the system" and some spare time on their hands.
Build in as much security as possible early on, because you'll need far more than you expect. During the 2012 Presidential Election, I was asked to pre-test 41 online voting sites, and was able to break 39 of them within the first 24 hours (6 of them within 1 hour). Be overly cautious. Know how attackers can get in, not just using "normal" mechanisms. Don't publish information about which technologies you're using, even in the code. Seeing "Rails-isms" anywhere in the HTML or Javascript code (or even the URL pathnames) will immediately give the attacker an enormous edge in defeating your safety mechanisms. Use obscurity to your advantage, and use security everywhere that you can.
NOTE: Checking the request.referer is like putting a padlock on a bank vault: it'll keep out those that are easily dissuaded, but won't even slow down the determined individual.
What you are trying to prevent here is basically cross-site request forgery. As Michael correctly pointed out, checking the Referer header will buy you nothing.
A popular counter-measure is to give each user an individual one-time token that is sent with each form and stored in the user's session. If, on submit, the submitted value and the stored value do not match, the request is disgarded. Luckily for you, RoR seems to ship such a feature. Looks like a one-liner indeed.
We have realized that this URL http://Keyword:redacted#example.com/ redirects to http://example.com/ when copied and pasted into the browser's address bar.
As far as I understand this might be used in some ftp connections but we have no such use on our website. We are suspecting that we are targeted by an attack and have been warned by Google that we are passing PII (mostly email addresses) in our URL requests to their Google Adsense network. We have not been able to find the source, but we have been warned that the violation is in the form of http://Keyword:redacted#example.com/
How can we stop this from happening?
What URL redirect method we can use to not accept this and return an error message?
FYI I experienced a similar issue for a client website and followed up with Adsense support. The matter was escalated to a specialist team who investigated and determined that flagged violations with the format http://Keyword:redacted#example.com/ will be considered false positives. I'm not sure if this applies to all publishers or was specific to our case, but it might be worth following up with Adsense support.
There is nothing you can do. This is handled entirely by your browser long before it even thinks about "talking" to your server.
That's a strange URL for people to copy/paste into the browser's address bar unless they have been told/trained to do so. Your best bet is to tell them to STOP IT! :-)
I suppose you could look at the HTTP Authorization Headers and report an error if they come in populated... (This would $_SERVER['PHP_AUTH_USER'] in PHP.) I've never looked at these values when the header doesn't request them, so I'm not sure if it would work or not...
The syntax http://abc:def#something.com means you're sending userid='abc', password='def' as basic authentication parameters. Your browser will pull out the userid & password and send them along as authentication information, leaving the url without them.
As Peter Bowers mentioned, you could check the authorization headers and see if they're coming in that way, but you can't stop others from doing it if they want. If it happens a lot then I'd suspect that somewhere there's a web form asking users to enter their user/password and it's getting encoded that way. One way to sleuth it out would be to see if you can identify someone by the userid specified.
Having Keyword:redacted sounds odd. It's possible Google Adsense changed the values to avoid including confidential info.
I'm having some issues with Google Analytics URL parameters. Prviously I've built URLs with the Google Analytics URL Builder. these have enabled me to track where visitors to my site have been coming from, how successful various marketing campaigns have been etc.
Recently, I've started using another tag in the URL, one which has nothing to do with Google Analytics, but acts to alter the telephone number on my site when the visitor arrives on it. For example, I'll add &ctcc=adwords onto the end of my tracking URL, and a specified phone number will appear on my site when the user comes through so I can track how many calls my adwords spend has generated.
However, when I've been using this ctcc code, Google Analytics no longer seems to be tracking the traffic numbers to my site :(
Any idea how I can incorporate the two parameters into the URl, and ensure that they both work as expected?
Thanks in advance
It looks like this is a problem with how your server is redirecting traffic with a ctcc query parameter. Look at the following request and its response headers:
So the ctcc parameter is used in some server side tracking (as best as I can tell), and the server is set up to redirect & strip ctcc whenever it gets a request with ctcc. Not being familiar with the system in use, I can't provide details, but you need to reconfigure the redirects to stop changing & into ;. It's the replacement of ampersands with semicolons that is messing up your GA data.
is it possible to submit forms with yahoo pipes?
i basically need to log in somewhere, and get some stuff from the members area of a website into a feed.
Although this is not exactly programming related... I guess it is close enough.
No, logging into somewhere is impossible with Yahoo Pipes. Sending the username/password isn't even the only problem here.
The real problem is that most, if not all, web sites that require a log-in depend on a session cookie or something similar. Yahoo pipes can do a GET request, and that's about it. Even if it was possible to send your user name/password in the URL, you would not be able to use the session cookie, so subsequent requests would fail.
So... If you have access to a hosted web site somewhere: Write a small proxy script (in PHP or whatever is available) that does the login and fetches the data. Let Yahoo pipes read from your proxy page. But if you are that far, you can just as well produce RSS format right away. ;-)
I did a pipe that can log in and extract info. is working ok on a simple web form using POST.