I have some trouble with my Jersey client (2.14).
I sent requests to my server (Jetty 8.1) every 100 ms. For 10 seconds there is no problems but every 10 seconds I have a request which remains blocked 2 seconds...
The trouble doesn't seem to come from jetty ...
Somebody got similar issue ?
private class PostJob implements Runnable {
#Override
public void run() {
while (start) {
try {
final HttpData take = daQueu.take();
final String path = take.path();
final Entity<Object> arg = take.arg();
for (final Map.Entry<String, ClientHandler> e : clients.entrySet()) {
final ClientHandler clientHandler = e.getValue();
long currentTimeMillis = System.currentTimeMillis();
Invocation.Builder request = null;
if (REQUEST_CACHE.containsKey(path)) {
request = REQUEST_CACHE.get(path);
} else {
LOGGER.debug("build new request");
request = clientHandler.client.target(path).request(MediaType.APPLICATION_JSON_TYPE);
REQUEST_CACHE.put(path, request);
}
Response post = request.post(arg);
LOGGER.debug("Execution delay for sending: {}", System.currentTimeMillis() - currentTimeMillis);
post.close();
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
I'm pretty sure that it doesn't come from Jetty Server... (but ....).
My server is embedded in Karaf container and is started by Pax-web feature.
Following a part of my server code which register my servlets:
final ResourceConfig restServletConf = new ResourceConfig()
.register(JacksonFeature.class).register(MultiPartFeature.class).register(SseFeature.class).register(JsonProcessingFeature.class);
restServletConf.registerInstances(restResources);
final HttpServlet restServlet = new ServletContainer(restServletConf);
try {
webContainer.registerServlet(conf.servletPath, restServlet, null, restService);
logService.log(LogService.LOG_INFO, "Serving JAX-RS resources from " + conf.servletPath);
// Servlet filters must be started after the REST servlet
// (constraint from PAX-Web's WebContainer API).
restService.start();
} catch (Exception e) {
logService.log(LogService.LOG_ERROR, "Failed to start JAX-RS servlet", e);
}
Related
My question is, behind the scene, for element-wise Beam DoFn (ParDo), how does the Cloud Dataflow parallel workload? For example, in my ParDO, I send out one http request to an external server for one element. And I use 30 workers, each has 4vCPU.
Does that mean on each worker, there will be 4 threads at maximum?
Does that mean from each worker, only 4 http connections are necessary or can be established if I keep them alive to get the best performance?
How can I adjust the level of parallelism other than using more cores or more workers?
with my current setting (30*4vCPU worker), I can establish around 120 http connections on the http server. But both server and worker has very low resource usage. basically I want to make them work much harder by sending out more requests out per second. What should I do...
Code Snippet to illustrate my work:
public class NewCallServerDoFn extends DoFn<PreparedRequest,KV<PreparedRequest,String>> {
private static final Logger Logger = LoggerFactory.getLogger(ProcessReponseDoFn.class);
private static PoolingHttpClientConnectionManager _ConnManager = null;
private static CloseableHttpClient _HttpClient = null;
private static HttpRequestRetryHandler _RetryHandler = null;
private static String[] _MapServers = MapServerBatchBeamApplication.CONFIG.getString("mapserver.client.config.server_host").split(",");
#Setup
public void setupHttpClient(){
Logger.info("Setting up HttpClient");
//Question: the value of maxConnection below is actually 10, but with 30 worker machines, I can only see 115 TCP connections established on the server side. So this setting doesn't really take effect as I expected.....
int maxConnection = MapServerBatchBeamApplication.CONFIG.getInt("mapserver.client.config.max_connection");
int timeout = MapServerBatchBeamApplication.CONFIG.getInt("mapserver.client.config.timeout");
_ConnManager = new PoolingHttpClientConnectionManager();
for (String mapServer : _MapServers) {
HttpHost serverHost = new HttpHost(mapServer,80);
_ConnManager.setMaxPerRoute(new HttpRoute(serverHost),maxConnection);
}
// config timeout
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(timeout)
.setConnectionRequestTimeout(timeout)
.setSocketTimeout(timeout).build();
// config retry
_RetryHandler = new HttpRequestRetryHandler() {
public boolean retryRequest(
IOException exception,
int executionCount,
HttpContext context) {
Logger.info(exception.toString());
Logger.info("try request: " + executionCount);
if (executionCount >= 5) {
// Do not retry if over max retry count
return false;
}
if (exception instanceof InterruptedIOException) {
// Timeout
return false;
}
if (exception instanceof UnknownHostException) {
// Unknown host
return false;
}
if (exception instanceof ConnectTimeoutException) {
// Connection refused
return false;
}
if (exception instanceof SSLException) {
// SSL handshake exception
return false;
}
return true;
}
};
_HttpClient = HttpClients.custom()
.setConnectionManager(_ConnManager)
.setDefaultRequestConfig(requestConfig)
.setRetryHandler(_RetryHandler)
.build();
Logger.info("Setting up HttpClient is done.");
}
#Teardown
public void tearDown(){
Logger.info("Tearing down HttpClient and Connection Manager.");
try {
_HttpClient.close();
_ConnManager.close();
}catch (Exception e){
Logger.warn(e.toString());
}
Logger.info("HttpClient and Connection Manager have been teared down.");
}
#ProcessElement
public void processElement(ProcessContext c) {
PreparedRequest request = c.element();
if(request == null)
return;
String response="{\"my_error\":\"failed to get response from map server with retries\"}";
String chosenServer = _MapServers[request.getHardwareId() % _MapServers.length];
String parameter;
try {
parameter = URLEncoder.encode(request.getRequest(),"UTF-8");
} catch (UnsupportedEncodingException e) {
Logger.error(e.toString());
return;
}
StringBuilder sb = new StringBuilder().append(MapServerBatchBeamApplication.CONFIG.getString("mapserver.client.config.api_path"))
.append("?coordinates=")
.append(parameter);
HttpGet getRequest = new HttpGet(sb.toString());
HttpHost host = new HttpHost(chosenServer,80,"http");
CloseableHttpResponse httpRes;
try {
httpRes = _HttpClient.execute(host,getRequest);
HttpEntity entity = httpRes.getEntity();
if(entity != null){
try
{
response = EntityUtils.toString(entity);
}finally{
EntityUtils.consume(entity);
httpRes.close();
}
}
}catch(Exception e){
Logger.warn("failed by get response from map server with retries for " + request.getRequest());
}
c.output(KV.of(request, response));
}
}
Yes, based on this answer.
No, you can establish more connections. Based on my answer, you can use a async http client to have more concurrent requests. As this answer also describes, you need to collect the results from these asynchronous calls and output it synchronously in any #ProcessElement or #FinishBundle.
See 2.
Since your resource usage is low, it indicates that the worker spends most of its time waiting for a response. I think with the described approach above, you can utilize your resources far better and you can achieve the same performance with far less workers.
This is the code and I am recieving the error 401: Authentication Error
public class Server {
// initialize socket and input stream
private Socket socket = null;
private ServerSocket server = null;
private DataInputStream in = null;
public void tweet() throws TwitterException {
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setDaemonEnabled(true).setOAuthConsumerKey("......")
.setOAuthConsumerSecret("......")
.setOAuthAccessToken("......")
.setOAuthAccessTokenSecret(".....");
TwitterFactory tf = new TwitterFactory();
twitter4j.Twitter twitter = tf.getInstance();
List status = twitter.getHomeTimeline();
for (Status st : status) {
System.out.println(st.getUser().getName() + "---- Tweets----" + st.getText());
}
}
// constructor with port
public Server(int port) throws TwitterException {
// starts server and waits for a connection
try {
server = new ServerSocket(port);
System.out.println("Server started");
System.out.println("Waiting for a client ...");
socket = server.accept();
System.out.println("Client accepted");
// takes input from the client socket
in = new DataInputStream(new BufferedInputStream(socket.getInputStream()));
String line = "";
// reads message from client until "Over" is sent
while (!line.equals("Over")) {
try {
line = in.readUTF();
System.out.println(line);
if (line.equalsIgnoreCase("Data")) {
tweet();
}
} catch (IOException i) {
System.out.println(i);
}
}
System.out.println("Closing connection");
// close connection
socket.close();
in.close();
} catch (IOException i) {
System.out.println(i);
}
}
public static void main(String args[]) throws TwitterException {
Server server = new Server(5000);
}
}
Please make sure that the tokens are valid.
Then, you could try enabling system proxies like so:
System.setProperty("java.net.useSystemProxies", "true");
I'm trying to understand the correct way to use the Flume RpcClient in a multithreaded application. Information I have found so far indicates that the components are thread safe, but the example in the Flume documentation clouds the issue when it comes to error handling. This code:
public void sendDataToFlume(String data) {
// Create a Flume Event object that encapsulates the sample data
Event event = EventBuilder.withBody(data, Charset.forName("UTF-8"));
// Send the event
try {
client.append(event);
} catch (EventDeliveryException e) {
// clean up and recreate the client
client.close();
client = null;
client = RpcClientFactory.getDefaultInstance(hostname, port);
// Use the following method to create a thrift client (instead of the above line):
// this.client = RpcClientFactory.getThriftInstance(hostname, port);
}
}
If more then one thread calls this method, and the exception is thrown, then there will be a problem as multiple threads try and recreate the client in the exception handler.
Is the intent of the SDK that it should only be used by a single thread? Should this method be synchronized, as it appears to be in the log4jappender that is part of the Flume source? Should I put this code in its own worker and pass it events via a queue?
Does anyone have an example of RpcClient being used by more then one thread (included the error condition)?
Would I be better off using the "embedded agent"? Is that multithread friendly?
With the embedded agent, you get the same case except you don't know what to do:
try {
agent.put(event);
} catch (EventDeliveryException e) {
// ???
}
You could stop the agent, and restart it - but you would need a synchronized block (or a ReentrantReadWriteLock, to not block thread while "reading" the client field). But since I'm not a Flume expert, I can't tell you which one is better.
Example:
class MyClass {
private final ReentrantReadWriteLocklock;
private final Lock readLock;
private final Lock writeLock;
private RpcClient client;
private final String hostname;
private final Integer port;
// Constructor
MyClass(String hostname, Integer port) {
this.hostname = Objects.requireNonNull(hostname, "hostname");
this.port = Objects.requireNonNull(port, "port");
this.lock = new ReentrantReadWriteLock();
this.readLock = this.lock.readLock();
this.writeLock = this.lock.writeLock();
this.client = buildClient();
}
private RpcClient buildClient() {
return RpcClientFactory.getDefaultInstance(hostname, port);
}
public void sendDataToFlume(String data) {
// Create a Flume Event object that encapsulates the sample data
Event event = EventBuilder.withBody(data, Charset.forName("UTF-8"));
// Send the event
readLock.lock(); // lock for reading 'client'
try {
try {
client.append(event);
} catch (EventDeliveryException e) {
writeLock.lock(); // lock for reading/writing client
try {
// clean up and recreate the client
client.close();
client = null;
client = buildClient();
} finally {
writeLock.unlock();
}
}
} finally {
readLock.unlock();
}
}
}
Beside, the example will lose the event because it is not sent back. Some kind of loop + a max retry would probably do the trick:
int i = 0;
for (; i < maxRetry; ++i) {
try {
client.append(event);
break;
} catch (EventDeliveryException e) {
// clean up and recreate the client
client.close();
client = null;
client = RpcClientFactory.getDefaultInstance(hostname, port);
// Use the following method to create a thrift client (instead of the above line):
// this.client = RpcClientFactory.getThriftInstance(hostname, port);
}
}
if (i == maxRetry) {
logger.error("flume client is offline, loosing events {}", event);
}
That's the idea, but I don't think that should be the task of the user (eg: us), but an option in the client or the agent to store event that could not be processed due to such errors.
Background:
I'm attempting to add some level fault tolerance to an application that uses Apache Commons.net FTPSClient to transfer files. If the connection between the client and server fails, I'd like to capture the produced exception/return code, log the details, and attempt to reconnect/retry the transfer.
What works:
The retrieveFile() method. If the connection fails, (i.e. I disable the server's public interface), I receive a CopyStreamException caused by a SocketTimeoutException after the amount of time I specified as the timeout.
What doesn't work:
The storeFile() method. If I initiate a transfer via storeFile() and disable the server's public interface, the storeFile() method blocks/hangs indefinitely with out throwing any exceptions.
Here is a simple app that hangs if the connection is terminated:
public class SmallTest {
private static Logger log = Logger.getLogger(SmallTest.class);
/**
* #param args
* #throws IOException
*/
public static void main(String[] args) throws IOException {
FTPSClient client = new FTPSClient(true);
FTPSCredentials creds = new FTPSCredentials("host", "usr", "pass",
"/keystore/ftpclient.jks", "pass",
"/keystore/rootca.jks");
String file = "/file/jdk-7u21-linux-x64.rpm";
String destinationFile = "/jdk-7u21-linux-x64.rpm";
client.setTrustManager(TrustManagerUtils.getValidateServerCertificateTrustManager());
client.setKeyManager(creds.getKeystoreManager());
client.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out), true));
client.setCopyStreamListener(createListener());
client.setConnectTimeout(5000);
client.setDefaultTimeout(5000);
client.connect(creds.getHost(), 990);
client.setSoTimeout(5000);
client.setDataTimeout(5000);
if (!FTPReply.isPositiveCompletion(client.getReplyCode())) {
client.disconnect();
log.error("ERROR: " + creds.getHost() + " refused the connection");
} else {
if (client.login(creds.getUser(), creds.getPass())) {
log.debug("Logged in as " + creds.getUser());
client.enterLocalPassiveMode();
client.setFileTransferMode(FTP.BLOCK_TRANSFER_MODE);
client.setFileType(FTP.BINARY_FILE_TYPE);
InputStream inputStream = new FileInputStream(file);
log.debug("Invoking storeFile()");
if (!client.storeFile(destinationFile, inputStream)) {
log.error("ERROR: Failed to store " + file
+ " on remote host. Last reply code: "
+ client.getReplyCode());
} else {
log.debug("Stored the file...");
}
inputStream.close();
client.logout();
client.disconnect();
} else {
log.error("Could not log into " + creds.getHost());
}
}
}
private static CopyStreamListener createListener(){
return new CopyStreamListener(){
private long megsTotal = 0;
#Override
public void bytesTransferred(CopyStreamEvent event) {
bytesTransferred(event.getTotalBytesTransferred(), event.getBytesTransferred(), event.getStreamSize());
}
#Override
public void bytesTransferred(long totalBytesTransferred,
int bytesTransferred, long streamSize) {
long megs = totalBytesTransferred / 1000000;
for (long l = megsTotal; l < megs; l++) {
System.out.print("#");
}
megsTotal = megs;
}
};
}
Is there any way to make the connection ACTUALLY timeout?
SW Versions:
Commons.net v3.3
Java 7
CentOS 6.3
Thanks in advance,
Joe
I ran into this same problem, and I think that I was able to get something that seems to work with the desired timeout behavior when I unplug the ethernet cable on my laptop.
I use 'storeFileStream' instead of 'storeFile', and then use 'completePendingCommand' to finish the transfer. You can check the Apache commons docs for 'completePendingCommand' to see an example of this kind of transfer. It took about 15 mins for it to timeout for me. One other thing: the aforementioned docs include calling 'isPositiveIntermediate' to check for an error, but this wasn't working. I replaced it with 'isPositivePreliminary' and now it seems to work. I'm not sure if that's actually correct, but it's the best I've found so far.
Can we call a webservice from the scheduled periodic task class firstly, if yes,
Am trying to call a webservice method with parameters in scheduled periodic task agent class in windows phone 7.1. am getting a null reference exception while calling the method though am passing the expected values to the parameters for the webmethod.
am retrieving the id from the isolated storage.
the following is my code.
protected override void OnInvoke(ScheduledTask task)
{
if (task is PeriodicTask)
{
string Name = IName;
string Desc = IDesc;
updateinfo(Name, Desc);
}
}
public void updateinfo(string name, string desc)
{
AppSettings tmpSettings = Tr.AppSettings.Load();
id = tmpSettings.myString;
if (name == "" && desc == "")
{
name = "No Data";
desc = "No Data";
}
tservice.UpdateLogAsync(id, name,desc);
tservice.UpdateLogCompleted += new EventHandler<STservice.UpdateLogCompletedEventArgs>(t_UpdateLogCompleted);
}
Someone please help me resolve the above issue.
I've done this before without a problem. The one thing you need to make sure of is that you wait until your async read processes have completed before you call NotifyComplete();.
Here's an example from one of my apps. I had to remove much of the logic, but it should show you how the flow goes. This uses a slightly modified version of WebClient where I added a Timeout, but the principles are the same with the service that you're calling... Don't call NotifyComplete() until the end of t_UpdateLogCompleted
Here's the example code:
private void UpdateTiles(ShellTile appTile)
{
try
{
var wc = new WebClientWithTimeout(new Uri("URI Removed")) { Timeout = TimeSpan.FromSeconds(30) };
wc.DownloadAsyncCompleted += (src, e) =>
{
try
{
//process response
}
catch (Exception ex)
{
// Handle exception
}
finally
{
FinishUp();
}
};
wc.StartReadRequestAsync();
}
private void FinishUp()
{
#if DEBUG
try
{
ScheduledActionService.LaunchForTest(_taskName, TimeSpan.FromSeconds(30));
System.Diagnostics.Debug.WriteLine("relaunching in 30 seconds");
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine(ex.ToString());
}
#endif
NotifyComplete();
}