I am new to Data Distributive Service.I am using PrismTech products for DDS. I have vortex lite in my network. To interact with the vortex gateway in public cloud , i am using vortex fog service. But was not able to establish interaction. Can any one please provide input for the same.
I have a DDS subsystem running on my network , data from it needs to be shared to the vortex gateway running in the cloud, for this purpose i am try to use Vortex fog . IN vortex fog configuration i have mentioned the public ip of the cloud server. I have a vortex gateway subscriber job running in the cloud to receive the dds data from the subsystem running in my network.
Fog service is running in the LAN behind the NAT . I had set below configurations for running fog service
fog.cluster.id=LAN1
fog.user.network.interface=eth1
fog.routing.network.interface=eth1
fog.services.network.interface=eth1
fog.services.tcp.peers=<public ip of cloud server>:7400
fog.externalNetworkAddresses=none
In cloud server i am running the vortex gateway subscribing to different topics.
Could you please correct/guide me to solve this issue
It is hard to give you a concrete answer as I don't have the details of your configuration. That said, let me try to give you some hint that may guide you toward the resolution of your problem.
Are there any applications subscribing for data on the Cloud? Notice that in Vortex data only flows if there is an interest arising. Otherwise no data is sent across the network -- that would just be a waste of precious resources. Beware that even if you have applications sharing data within a Fog but no applications subscribing to data "outside" the Fog, data won't be pushed out by Vortex-Fog. Once again, data flows only where there is an interest.
I assume that you are using Fog because you have an entire sub-system, i.e. several DDS applications, whose data needs to be efficiently shared with the cloud while maintaining multicast communication on the sub-system. If this is not the case, then you can simply configure Lite and the Gateway to use TCP/IP and have them talk directly. That would probably be the simplest deployment.
To ensure that you don't have any specific problems with your network set-up have you tried to run two Lite applications that use TCP/IP and communicate through our public Vortex Cloud instance available at demo-eu.prismtech.com or demo-us.prismtech.com?
If you post your configuration files I may be able to give you more insights.
HTH.
A+
Related
I have a microservices based software architecture.
There is a php application which orchestrates the communication among microservices and the application's whole logic.
I need to simulate the communication between microservices as a graph.
There will be edges with weights , which will represent the affinities between microservices.
I am searching for a tool in order to collect all messages and their size.
I have read that there are distibuted tracing systems like Zipkin which i have already deployed, and could accomplish this task.
But, i cannot find how to collect the messages i want.
This is the php library i used for the instrumentation of my app
[https://github.com/openzipkin/zipkin-php]
Any ideas about other tools or how to use Zipkin differently to achieve my goal?
Let me add to this thread my three bits. Speaking of Envoy, yes, when attached to your application it adds a lot of useful features from observability bucket, e.g. network level statistics and tracing.
Here is the question, have you considered running your legacy apps inside service mesh, like Istio ?.
Istio simplifies deployment and configuration of Envoy for you. It injects sidecar container (istio-proxy, in fact Envoy instance) to your Pod application, and gives you these extra features like a set of service metrics out of the box*.
Example: Stats produced by Envoy in Prometheus format, like istio_request_bytes are visualized in Kiali Metrics dashboard for inbound traffic as request_size (check screenshot)
*as mentioned by #David Kruk, you still needs to have Prometheus server deployed in your cluster to be able to pull these metrics to Kiali dashboards.
You can learn more about Istio here. There is also a dedicated section on how to visualize metrics collected by Istio (e.g. request size).
I'm considering setting up Prometheus monitoring stack using federation to handle issues with poor connectivity. My use case is the following:
I have N separate small setups on-prem, each consisting of several machines and docker containers running on them
Each of those setups is connected to the cloud, but connection is poor and can be lost sometimes
I need to have a single prometheus instance in the cloud which would aggregate data from all those "small setup" on-prem
My idea was that local prom servers will scrape my jobs/machines etc and then using federation those metrics will land in the "central" prom instance.
Now, I'm not sure what will happen if I loose connectivity between cloud and on-prem. Am I going to download all samples from local prometheus servers, once connectivity is back? Or the central server will never learn about what happened during such network outage?
I love using Prometheus for monitoring and alerting. Until now, all my targets (nodes and containers) lived on the same network as the monitoring server.
But now I'm facing a scenario, where we will deploy our application stack (as a bunch of Docker containers) to several client machines in thier networks. Nearly all of the clients networks are behind a firewall or NAT. So scraping becomes quite difficult.
As we're still accountable for our stack, I'd like to have a central montioring server, altering and dashboards.
I was wondering what could be the best architecture if want to implement it with Prometheus, but I couldn't find any convincing approaches. My ideas so far:
Use a Pushgateway on our side and push all data out of the client networks. As the docs state, it's not intended that way: https://prometheus.io/docs/practices/pushing/
Use a federation setup (https://prometheus.io/docs/prometheus/latest/federation/): Place a Prometheus server in every client network behind a reverse proxy (to enable SSL and authentication) and aggregate relevant metricts there. Open/forward just a single port for federation scraping.
Other more experimental setups, such as SSH Tunneling (e.g. here https://miek.nl/2016/february/24/monitoring-with-ssh-and-prometheus/) or VPN!?
Thank you in advance for your help!
Nobody posted an answer so I will try to give my opinion on the second choice because that's what I think I would do in your situation.
The second setup seems the most flexible, you have access to the datas and only need to open one port on for the federating server, so it should still be secure.
One other bonus of this type of setup is that even if the firewall stop working for a reason or another, you will still have a prometheus scraping, you will have an alert because you won't be able to access the server(s) but when the connexion comes again you will have all the datas. You won't have a hole in the grafana dashboards because there was no datas, apart during the incident.
The issue with this setup is the fact that you need to maintain a number of server equivalent to the number of networks. A solution for this would be to have a packer image or maybe an ansible playbook to deploy.
My question is related to microservices & service discovery of a service which is spread between several hosts.
The setup is as follows:
2 docker hosts (host A & host B)
a Consul server (service discovery)
Let’s say that I have 2 services:
service A
service B
Service B is deployed 10 times (with random ports): 5 times on host A and 5 times on host B.
When service A communicates with service B, for example, it sends a request to serviceB.example.com (hard coded).
In order to get an IP and a port, service A should query the Consul server for an SRV record.
It will get 10 ip:port pairs, for which the client should apply some load-balancing logic.
Is there a simpler way to handle this without me developing a client resolver (+LB) library for that matter ?
Is there anything like that already implemented somewhere ?
Am I doing it all wrong ?
There are a few options:
Load balance on client as you suggest for which you'll either need to find a ready-build service discovery library that works with SRV records and handles load balancing and circuit breaking. Another answer suggested Netflix' ribbon which I have not used and will only be interesting if you are on JVM. Note that if you are building your own, you might find it simpler to just use Consul's HTTP API for discovering services than DNS SRV records. That way you can "watch" for changes too rather than caching the list and letting it get stale.
If you don't want to reinvent that particular wheel, another popular and simple option is to use a HAProxy instance as the load balancer. You can integrate it with consul via consul-template which will automatically watch for new/failed instances of your services and update LB config. HAProxy then provides robust load balancing and health checking with a lot of options (http/tcp, different balancing algorithms, etc). One possible setup is to have a local HAProxy instance on each docker host and a fixed port assigned statically to each logical service (can store it in Consul KV) so you connect to localhost:1234 for service A for example and localhost:2345 for service B. Local instance means you don't pay for extra round trip to loadbalancer instance then to the actual service instance but this might not be an issue for you.
I suggest you to check out Kontena. It will solve this kind of problem out of the box. Every service will have an internal DNS that you can use in communication between services. Kontena has also built-in load balancer that is very easy to use making it very easy to create and scale micro services.
There are also lot's of built-in features that will help developing containerized applications, like private image registry, VPN access to running services, secrets management, stateful services etc.
Kontena is open source project and the code is visible on Github
If you look for a minimal setup, you can wrap the values you receive from Consul via ribbon, Netflix' client based load balancer.
You will find it as a module for Spring Cloud.
I didn't find an up-to-date standalone example, only this link to chrisgray's dropwizard-consul implementation that is using it in a Dropwizard context. But it might serve as a starting point for you.
I would like to use the apache-jena-fuseki server to provide an ontology within a local network environment. And as such it should be editable from all machines in the LAN ideally via a webinterface (for simplicity).
The built-in webinterface of apache-jena-fuseki is doing that job well, BUT only from the machine running the server. Accessing the webinterface from other machines will somehow not provide the functionalities of managing (uploading & deleting) the datasets.
Is this a configuration issue? And, if yes, how and where should it be modified?
Any help on the matter is highly appreciated!
Cheers!