Load More data by using HTTPURLConnection in java - twitter

In my program , I read data by using HTTPURLConnection for example
URL url = new URL("https://twitter.com/search?q="+kw+"%20lang%3Aeng%20"+"&src=typd&lang=en");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
if (conn.getResponseCode() != 200) {
throw new RuntimeException("Failed : HTTP error code : "
+ conn.getResponseCode());
}
String line;
StringBuilder builder = new StringBuilder();
BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
while ((line = reader.readLine()) != null) {
builder.append(line);
builder.append("\n");
}
String html = builder.toString();
The problem is only reading data on response page , but I need to read all data that can be retrieved
How I can Load more data in result page

Related

add parameters in JSON FORM to HttpURLConnection using POST

Well the best vote for adding parameters to HttpURLConnection is in this post
But the answer of it is explain about how to adding parameter that have the form something like this
--> "username=usernameValue?password=passwordValue"<--
REMEMBER the parameter can have the form like JSON Object and the answer form the link can make you lost of direction if you dont know about JSON Parameter for POST.
Then this is the best answer from me that i can provide.
URL url = new URL("http://yoururl.com");
HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
conn.setReadTimeout(10000);
conn.setConnectTimeout(15000);
conn.setRequestMethod("POST");
conn.setDoInput(true);
conn.setDoOutput(true);
JSONObject urlParameter = new JSONObject();
urlParameter.put("username", usernameValue);
urlParameter.put("password", passwordValue);
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(urlParameter.toString());
wr.flush();
wr.close();
int responseCode = conn.getResponseCode();
StringBuilder sb;
sb = new StringBuilder();
if (responseCode == HttpURLConnection.HTTP_OK) {
String line;
BufferedReader br = new BufferedReader(new InputStreamReader(conn.getInputStream()));
while ((line = br.readLine()) != null) {
sb.append(line).append("\n");
}
} else {
System.out.println(conn.getResponseMessage());
}
String result = sb.toString();
Please enlighten me if you have better solution. TY

Java client hangs on getResponseCode() when attempt to upload small size files

I have a client/server application with which the client is able to upload files to the server and then they are stored in a DB. I followed the instructions found in the following article.
It works fine with files bigger than 1MB, but it doesn't with smallers: the client hangs on getResponseCode().
Here there is my specific client code:
final String UPLOAD_URL = "http://localhost:8080/myApp/myUploadAction"
// takes file path from first program's argument
String filePath = "/home/user/myCSV.csv"
File uploadFile = new File(filePath);
System.out.println("File to upload: " + filePath);
// creates a HTTP connection
URL url = new URL(UPLOAD_URL);
HttpURLConnection httpConn = (HttpURLConnection) url.openConnection();
httpConn.setUseCaches(false);
httpConn.setDoOutput(true);
httpConn.setRequestMethod("POST");
// sets file name as a HTTP header
httpConn.setRequestProperty("fileName", uploadFile.getName());
// opens output stream of the HTTP connection for writing data
OutputStream outputStream = httpConn.getOutputStream()
// Opens input stream of the file for reading data
InputStream inputStream = new FileInputStream(uploadFile);
System.out.println("Start writing data...");
byte[] buffer = new byte[4096];
int bytesRead;
while ((bytesRead = inputStream.read(buffer)) != -1) {
println bytesRead
outputStream.write(buffer, 0, bytesRead);
}
System.out.println("Data was written.");
outputStream.close();
inputStream.close();
// always check HTTP response code from server
int responseCode = httpConn.getResponseCode();
if (responseCode == HttpURLConnection.HTTP_OK) {
// reads server's response
BufferedReader reader = new BufferedReader(new InputStreamReader(
httpConn.getInputStream()));
String response = reader.readLine();
System.out.println("Server's response: " + response);
} else {
System.out.println("Server returned non-OK code: " + responseCode);
}
Here there is my server side code. The server is based on a Grails application which exposes the following controller to the client:
class UploaderController {
ReportService reportService
def uploadCSV() {
// get CSV file and NPL process name from HTTP header
String fileName = request.getHeader("fileName");
log.info('npl-getcolls-report: received request for ' + nplProcessName + ' - file uploading - ' + fileName)
// prints out all header values (if debug mode is active)
println("===== Begin headers =====")
Enumeration<String> names = request.getHeaderNames()
while (names.hasMoreElements()) {
String headerName = names.nextElement()
println(headerName + " = " + request.getHeader(headerName))
}
println("===== End headers =====\n")
// open input stream of the request for read incoming data
InputStream inputStream = request.getInputStream()
// store CSV into DB
reportService.storeCSV(fileName, inputStream.getBytes())
log.info("data received and stored correctly in DB");
inputStream.close();
def responseData = [
'status': 'OK'
]
render responseData as JSON
}
}

get_collector_list request in java

trying the request by passing the surveyID as a parameter from java using following input string.
Getting following error
{"status": 3, "errmsg": "No JSON object could be decoded: line 1 column 0 (char 0)"}
String input ="{\"survey_id\": p_sSurveyID, \"fields\":[\"url\"]}"; -- not working
where as same is working fine if the surveyID is hard coded
String input ="{\"survey_id\":\"12345678\", \"fields\":[\"url\"]}"; -- working
Probably you are not concatenating the psSurveyID properly.
String input ="{\"survey_id\": p_sSurveyID, \"fields\":[\"url\"]}"; -- not working
should be
String input ="{\"survey_id\":"+ p_sSurveyID+",\"fields\":[\"url\"]}"; -- should work
System.out.println("p_sSurveyID --- " + p_sSurveyID);
try
{
List<NameValuePair> parameters = new ArrayList<NameValuePair>();
parameters.add(new NameValuePair("api_key",p_sApiKey));
URL url = new URL(createUrl(BASE_URL+COLLECTOR_LIST_ENDPOINT,parameters));
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", "application/json");
conn.setRequestProperty("Authorization", "bearer "+p_sAuthToken);
String input ="{\"survey_id\":"+ p_sSurveyID+",\"fields\":[\"url\"]}";
OutputStream os = conn.getOutputStream();
os.write(input.getBytes());
os.flush();
if (conn.getResponseCode() != HttpURLConnection.HTTP_OK) {
throw new RuntimeException("Failed : HTTP error code : " +conn.getResponseCode());
}
BufferedReader br = new BufferedReString input ="{\"survey_id\":"+ p_sSurveyID+",\"fields\":[\"url\"]}";
OutputStream os = conn.getOutputStream();
os.write(input.getBytes()); os.flush();
if (conn.getResponseCode() != HttpURLConnection.HTTP_OK) {
throw new RuntimeException("Failed : HTTP error code : " +
conn.getResponseCode());
}
BufferedReader br = new BufferedReader(new InputStreamReader( (conn.getInputStream())));
conn.disconnect();
ader(new InputStreamReader( (conn.getInputStream())));
conn.disconnect();

How can I download XML file to Document Directory of blackberry using programming?

I have link of XML file, how I want to download it in my local directory as application starts,,,,
I am working on blackberry application
How can I do it ?
Try this:
HttpConnection httpConnector = (HttpConnection) Connector.open(urlStr + ";interface=wifi");
httpConnector.setRequestMethod(HttpConnection.GET);
InputStream in = httpConnector.openInputStream();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(in));
StringBuffer content = new StringBuffer();
String line;
while ((line = bufferedReader.readLine()) != null) {
content.append(line + "\n");
}
This code will work if device has wifi connection. Of course, you should use try-catch block and you can use String instead of StringBuffer, it is up to you.

Retrieve GPS Location and send it to Web Server [duplicate]

i am developing and app for blackberry and i need to send a Http Post Request to my server. I'm using the simulator in order to test my app and i found this code in order to send request:
http://vasudevkamath.techfiz.com/general/posting-data-via-http-from-blackberry/
But i can't get it work, because it fails in this line:
int rc = _httpConnection.getResponseCode();
Any idea?
thanks
Here is a sample code on how to send a POST request:
HttpConnection c = (HttpConnection) Connector.open(url, Connector.READ_WRITE);
c.setRequestMethod(HttpConnection.POST);
OutputStream os = c.openOutputStream();
os.write(request.getBytes("UTF-8"));
os.flush();
os.close();
InputStream is = c.openInputStream();
Just make sure you use this code in a separate thread.
public static ResponseBean sendRequestAndReceiveResponse(String method, String absoluteURL, String bodyData, boolean readResponseBody)
throws IOException
{
ResponseBean responseBean = new ResponseBean();
HttpConnection httpConnection = null;
try
{
String formattedURL = absoluteURL + "deviceside=true;interface=wifi"; // If you are using WiFi
//String formattedURL = absoluteURL + "deviceside=false"; // If you are using BES
//String formattedURL = absoluteURL + "deviceside=true"; // If you are using TCP
if(DeviceInfo.isSimulator()) // if you are using simulator
formattedURL = absoluteURL;
httpConnection = (HttpConnection) Connector.open(formattedURL);
httpConnection.setRequestMethod(method);
if (bodyData != null && bodyData.length() > 0)
{
OutputStream os = httpConnection.openOutputStream();
os.write(bodyData.getBytes("UTF-8"));
}
int responseCode = httpConnection.getResponseCode();
responseBean.setResponseCode(responseCode);
if (readResponseBody)
{
responseBean.setBodyData(readBodyData(httpConnection));
}
}
catch (IOException ex)
{
System.out.println("!!!!!!!!!!!!!!! IOException in NetworkUtil::sendRequestAndReceiveResponse(): " + ex);
throw ex;
}
catch(Exception ex)
{
System.out.println("!!!!!!!!!!!!!!! Exception in NetworkUtil::sendRequestAndReceiveResponse(): " + ex);
throw new IOException(ex.toString());
}
finally
{
if (httpConnection != null)
httpConnection.close();
}
return responseBean;
}
public static StringBuffer readBodyData(HttpConnection httpConnection) throws UnsupportedEncodingException, IOException
{
if(httpConnection == null)
return null;
StringBuffer bodyData = new StringBuffer(256);
InputStream inputStream = httpConnection.openDataInputStream();
byte[] data = new byte[256];
int len = 0;
int size = 0;
while ( -1 != (len = inputStream.read(data)) )
{
bodyData.append(new String(data, 0, len,"UTF-8"));
size += len;
}
if (inputStream != null)
{
inputStream.close();
}
return bodyData;
}
I know this question is pretty old and OP probably solved it by now, but I've just run into the same problem and managed to fix it!
You need to append ;deviceside=true to your URL.
So for example, your URL will change from "http://example.com/directory/submitpost.php" to "http://example.com/directory/submitpost.php;deviceside=true".
I found this here: http://supportforums.blackberry.com/t5/Java-Development/Different-ways-to-make-an-HTTP-or-socket-connection/ta-p/445879
My POST request was timing out after 3 minutes when I did not have this (See My Comment), but it works fine with this appended to the url.
I would also recommend using ConnectionFactory. Here's some of my code:
Network.httpPost("http://example.com/directory/submitpost.php;deviceside=true", paramNamesArray, paramValsArray)
public static void httpPost(String urlStr, String[] paramName, String[] paramVal) throws Exception {
ConnectionFactory conFactory = new ConnectionFactory();
conFactory.setTimeLimit(1000);
HttpConnection conn = (HttpConnection) conFactory.getConnection(urlStr).getConnection();
conn.setRequestMethod(HttpConnection.POST);
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
StringBuffer sb = new StringBuffer();
for (int i = 0; i < paramName.length; i++) {
sb.append(paramName[i]);
sb.append("=");
sb.append(paramVal[i]);
sb.append("&");
}
byte[] postData = sb.toString().getBytes("UTF-8");
conn.setRequestProperty("Content-Length",new Integer(postData.length).toString());
OutputStream out = conn.openOutputStream();
out.write(postData);
//out.flush(); //Throws an Exception for some reason/Doesn't do anything anyways
out.close();
//This writes to our connection and waits for a response
if (conn.getResponseCode() != 200) {
throw new Exception(conn.getResponseMessage());
}
}
Not sure about the site you posted, but I've successfully used the sample ConnectionFactory code provided on the blackberry site.
http://supportforums.blackberry.com/t5/Java-Development/Sample-Code-Using-the-ConnectionFactory-class-in-a-BrowserField/ta-p/532860
Just make sure not to invoke the connection on the EventThread.
That's how you add parameters, Full answer is here:
StringBuffer postData = new StringBuffer();
httpConn = (HttpConnection) Connector.open("https://surveys2.kenexa.com/feedbacksurveyapi/login?");
httpConn.setRequestMethod(HttpConnection.POST);
postData.append("username="+username);
postData.append("&password="+pass);
postData.append("&projectcode="+projectid);
String encodedData = postData.toString();
httpConn.setRequestProperty("Content-Language", "en-US");
httpConn.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
httpConn.setRequestProperty("Content-Length",(new Integer(encodedData.length())).toString());
byte[] postDataByte = postData.toString().getBytes("UTF-8");
OutputStream out = httpConn.openOutputStream();
out.write(postDataByte);
out.close();
httpConn.getResponseCode();

Resources