Here's an excerpt from Live HTTP headers, I've replaced several values for anonymity.
POST blah/admin.php?module_id=1&action=update&id=129&pageNum=17&&eid=362 HTTP/1.1
Host: blah
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.12) Gecko/20101027 Firefox/3.6.12
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: blah
Cookie: blah
Content-Type: multipart/form-data; boundary=---------------------------21278813472729408841849703914
Content-Length: 5110
-----------------------------21278813472729408841849703914
Content-Disposition: form-data; name="MAX_FILE_SIZE"
300000000
This request has both GET and POST values. The script on the other end of this is PHP and expects certain values to be in the GET and others to be in the POST.
I know how to issue a GET
curl -G -d "key=val" "http://yadayadayada"
And I understand how to do a POST
curl -d "key=val" "http://yadayadayada"
curl -F "key=val" "http://yadayadayada"
But how do I mix the two in a single request? Every attempt I've made so far has ended in an error.
GET variables can be included in the URL. You just include the GET variables in the query string. For example, if you wanted to send a GET request with "username=fred" to www.example.com/index.php, you would send a simple GET request to "http://www.example.com/index.php?username=fred". So to answer your question, just use the POST method, but have the URL contain your GET data.
To clarify, GET and POST are HTTP request methods, not value types.
GET variables are called query string parameters. They are part of the URL, and can be included in any request.
POST variables are the contents of a urlencoded message body. These might also be sent with a PUT request.
Therefore, if you want to send both types of values, send the POST data as normal while explicitly writing your query string.
curl -d "key=val" "http://example.com?query_var=1"
Related
I am using pyshark to parse .pcap files specifically with HTTP packets. Unlike as in Wireshark, where it shows the \r\n\r\n bytes at the end of the HTTP header, pyshark does not show them and instead shows a single \r\n.
Is there any way to properly parse the HTTP layer of the packet to display the \r\n\r\n's?
If so, how? I have done a fair amount of searching through the web but the sources are limited and does not answer my question.
Also, with pyshark, the headers do not come in the same order as seen on Wireshark. Is there any reason to that as well?
Python code
#!/bin/env python3
import pyshark
packets = []
with pyshark.FileCapture('testing-mutillidae1.pcap') as capture:
for pkt in capture: # storing packets in list
packets.append(pkt)
print(packets[3]) # printing packet details of packet no. 4
HTTP header
I have included the full output of the packet on pastebin: https://pastebin.com/qxjxY6Hw . Since it is too long, I have added only the HTTP layer in this question
Layer HTTP:
GET /mutillidae/index.php?page=add-to-your-blog.php HTTP/1.1\r\n
Expert Info (Chat/Sequence): GET /mutillidae/index.php?page=add-to-your-blog.php HTTP/1.1\r\n
GET /mutillidae/index.php?page=add-to-your-blog.php HTTP/1.1\r\n
Severity level: Chat
Group: Sequence
Request Method: GET
Request URI: /mutillidae/index.php?page=add-to-your-blog.php
Request URI Path: /mutillidae/index.php
Request URI Query: page=add-to-your-blog.php
Request URI Query Parameter: page=add-to-your-blog.php
Request Version: HTTP/1.1
Host: 10.0.2.13\r\n
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0\r\n
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n
Accept-Language: en-US,en;q=0.5\r\n
Accept-Encoding: gzip, deflate\r\n
Referer: http://10.0.2.13/mutillidae/index.php\r\n
Connection: keep-alive\r\n
Cookie: showhints=0; PHPSESSID=511be46cfd6922ff8sqqhtqmbn\r\n
Cookie pair: showhints=0
Cache-Control: max-age=0\r\n
Full request URI: http://10.0.2.13/mutillidae/index.php?page=add-to-your-blog.php
HTTP request 1/1
\r\n
Upgrade-Insecure-Requests: 1\r\n
Cookie pair: PHPSESSID=511be46cfd6922ff8sqqhtqmbn
Here is the screenshot on my wireshark (I cannot post pictures yet)
As far as I understand, POST parameters can be sent using Content-Type only either multipart/form-data or x-www-form-urlencoded. Is that right?
I want to send the parameters wrapped in JSON. Is that possible?
Will the parameters in the request
POST /login HTTP/1.1
Host: localhost:8080
Content-Type: application/json
{"username": "test","password": "test"}
be identified as username=test&password=test?
I'm not a ruby or rails programmer, but I'm tasked with reverse engineering an API for an RoR app. My HTTP POST requests are failing a validation check, where this line is supposed to provide a specific piece of data:
value = #_request.env['HTTP_X_MY_TOKEN'];
From what little experience I have and searching I've done, it appears to be looking for an HTTP request header MY_TOKEN but I'm unsure if that's the case.
My current HTTP request looks like this:
POST /myapp HTTP/1.1
Host: website.com:80
Content-Type: application/json
Content-Length: 12
my post data
If that is the case, can I simply add it to my HTTP post request headers as follows:
POST /myapp HTTP/1.1
Host: website.com:80
Content-Type: application/json
MY_TOKEN: sometokentext
Content-Length: 12
my post data
If not, how do I fill this value during my HTTP POST request?
Sending X-MY-TOKEN should do the trick.
As a side note, prepending custom headers with X- is no longer recommended and deprecated according to RFC-6648:
Custom HTTP headers : naming conventions
Have asked the same question in Spring Data forum, but think that it is related to Neo4j API.The query is working fine when run within the webadmin.
The request/response, as shown below. An update: - works fine when the values are hard-coded and passed.
POST /db/data/cypher HTTP/1.1
Accept: application/json;stream=true
X-Stream: true
Content-Type: application/json
User-Agent: neo4j-rest-graphdb/0
Host: localhost:7000
Connection: keep-alive
Transfer-Encoding: chunked
b1
{"query":"START n=node:LAT_LONG('withinDistance:[{0},{1}, {2}]') match n<-[:address]-(location)<-[:CONTAINS]-(pol) return pol","params": {"2":50.0,"1":-74.598347,"0":39.274423}}
0
The exception that is received is:
HTTP/1.1 400 Bad Request
Content-Encoding: UTF-8
Content-Type: application/json; stream=true
Transfer-Encoding: chunked
Server: Jetty(6.1.25)
D70
{"exception":"BadInputException","fullname":"org.neo4j.server.rest.repr.BadInputException","stacktrace":["org.neo4j.server.rest.repr.RepresentationExceptionHandlingIterable.exceptionOnHasNext(RepresentationExceptionHandlingIterable.java:50)","org.neo4j.helpers.collection.ExceptionHandlingIterable$1.hasNext(ExceptionHandlingIterable.java:60)","org.neo4j.helpers.collection.IteratorWrapper.hasNext(IteratorWrapper.java:42)","org.neo4j.server.rest.repr.ListRepresentation.serialize(ListRepresentation.java:58)","org.neo4j.server.rest.repr.Serializer.serialize(Serializer.java:75)","org.neo4j.server.rest.repr.MappingSerializer.putList(MappingSerializer.java:61)","org.neo4j.server.rest.repr.CypherResultRepresentation.serialize(CypherResultRepresentation.java:57)","org.neo4j.server.rest.repr.MappingRepresentation.serialize(MappingRepresentation.java:42)","org.neo4j.server.rest.repr.OutputFormat$1.write(OutputFormat.java:144)"],"cause":{"exception":"NullPointerException","fullname":"java.lang.NullPointerException","stacktrace":["org.neo4j.gis.spatial.indexprovider.LayerNodeIndex.query(LayerNodeIndex.java:246)","org.neo4j.gis.spatial.indexprovider.LayerNodeIndex.query(LayerNodeIndex.java:289)","org.neo4j.cypher.internal.spi.gdsimpl.GDSBackedQueryContext$$anon$1.indexQuery(GDSBackedQueryContext.scala:87)","org.neo4j.cypher.internal.executionplan.builders.IndexQueryBuilder$$anonfun$getNodeGetter$2.apply(IndexQueryBuilder.scala:83)","org.neo4j.cypher.internal.executionplan.builders.IndexQueryBuilder$$anonfun$getNodeGetter$2.apply(IndexQueryBuilder.scala:81)","org.neo4j.cypher.internal.pipes.matching.MonoDirectionalTraversalMatcher.findMatchingPaths(MonodirectionalTraversalMatcher.scala:45)","org.neo4j.cypher.internal.pipes.TraversalMatchPipe$$anonfun$internalCreateResults$1.apply(TraversalMatchPipe.scala:38)","org.neo4j.cypher.internal.pipes.TraversalMatchPipe$$anonfun$internalCreateResults$1.apply(TraversalMatchPipe.scala:35)","scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)","scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)","org.neo4j.cypher.internal.ClosingIterator$$anonfun$hasNext$1.apply$mcZ$sp(ClosingIterator.scala:36)","org.neo4j.cypher.internal.ClosingIterator$$anonfun$hasNext$1.apply(ClosingIterator.scala:35)","org.neo4j.cypher.internal.ClosingIterator$$anonfun$hasNext$1.apply(ClosingIterator.scala:35)","org.neo4j.cypher.internal.ClosingIterator.failIfThrows(ClosingIterator.scala:86)","org.neo4j.cypher.internal.ClosingIterator.hasNext(ClosingIterator.scala:35)","org.neo4j.cypher.PipeExecutionResult.hasNext(PipeExecutionResult.scala:133)","scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)","scala.collection.convert.Wrappers$IteratorWrapper.hasNext(Wrappers.scala:29)","org.neo4j.helpers.collection.ExceptionHandlingIterable$1.hasNext(ExceptionHandlingIterable.java:58)","org.neo4j.helpers.collection.IteratorWrapper.hasNext(IteratorWrapper.java:42)","org.neo4j.server.rest.repr.ListRepresentation.serialize(ListRepresentation.java:58)","org.neo4j.server.rest.repr.Serializer.serialize(Serializer.java:75)","org.neo4j.server.rest.repr.MappingSerializer.putList(MappingSerializer.java:61)","org.neo4j.server.rest.repr.CypherResultRepresentation.serialize(CypherResultRepresentation.java:57)","org.neo4j.server.rest.repr.MappingRepresentation.serialize(MappingRepresentation.java:42)","org.neo4j.server.rest.repr.OutputFormat$1.write(OutputFormat.java:144)"]}}
0
The link to my question in Spring forum is BadInputException-for-a-neo4j-Custom-query
You cannot use parameters within the strings of an index query 'withinDistance:[{0},{1}, {2}]', you have to provide the whole index query as a parameter.
{"query":"START n=node:LAT_LONG({indexQuery})
match n<-[:address]-(location)<-[:CONTAINS]-(pol)
return pol",
"params": {"indexQuery": "withinDistance:[39.27442339.274423, -74.598347, 50.0 ]"} }
I'm been playing with calling my rails controller using an HTTP POST. I can get it to work with a curl command such as this, given a model named item and an attribute in that item called name:
curl -X POST -d "<item><name>myname</name></item>" -H "Content-Type: text/xml" http://localhost:3000/items.xml
What I'm curious about is how to make the same call using text instead of xml as my content type.... I tried:
curl -X POST -d "name=myname" http://localhost:3000/items.xml
but that seems to pass the 'name' parameter as the top level scope; so it doesn't end up in my params in the controller....
I'm a noob at this; just want to understand how to do it both ways....
Thanks!
Short answer, you need to
(a) send it as -H "Content-type: application/x-www-form-urlencoded".
(b) specify the parameters as Object[field]=value - for example User[name] would refer to the name field in some user object.
(c) [Not Required because you use curl] encode the parameters and POST/PUT them.
Encoding the parameters
The encoding is simple enough and although curl will do it for you, it's useful to understand it. I've quoted an extract from the W3C spec...
"
application/x-www-form-urlencoded
This is the default content type. Forms submitted with this content type must be encoded as follows:
Control names and values are escaped. Space characters are replaced by +', and then reserved characters are escaped as described in [RFC1738], section 2.2: Non-alphanumeric characters are replaced by%HH', a percent sign and two hexadecimal digits representing the ASCII code of the character. Line breaks are represented as "CR LF" pairs (i.e., `%0D%0A').
The control names/values are listed in the order they appear in the document. The name is separated from the value by =' and name/value pairs are separated from each other by&'.
"
Simple Example (New User Form)
The example below is how to send a simple "New User" form.
If I have fields in the user object for name, password, email etc, I specify them like this ...
user[firstname]=chris&user[login]=cmccauley&user[company_id]=8&user[email]=&user[surname]=mccauley
then curl will escape them to give ...
user%5Bfirstname%5D=chris&user%5Blogin%5D=cmccauley&user%5Bcompany_id%5D=8&user%5Bemail%5D=&user%5Bsurname%5D=mccauley
... before posting them like this extract from a wire dump ...
Wireshark dump
POST /users/19 HTTP/1.1
Host: localhost:3000
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042523 Ubuntu/9.04 (jaunty) Firefox/3.0.10
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://localhost:3000/users/19/edit
Cookie: _my_session=048d330143de668e027c8cd52654e8c5
Content-Type: application/x-www-form-urlencoded
Content-Length: 259
user%5Bfirstname%5D=chris&user%5Blogin%5D=cmccauley&user%5Bcompany_id%5D=8&user%5Bemail%5D=&user%5Bsurname%5D=mccauley&user_password=********&user%5Bjob_id%5D=14&user%5Bpassword%5D=dd793a64b74e108fcdc5d809040e24afcc21ad2c&authenticity_token=&id=19&_method=PUT
"Content-type: application/x-www-form-urlencoded" is already default with curl -d, no need for anything extra
-X POST is superfluous, as -d implies POST
-d does not URL encode the data. To get curl to do that for you, you need to use --data-urlencode instead