I have the following elastic-search container configuration in my test case
#Container
public static GenericContainer container = new GenericContainer<>("elasticsearch:7.7.0")
.withExposedPorts(9200,9300).withEnv("discovery.type","single-node")
.withNetwork(Network.newNetwork())
.withNetworkAliases("someNetwork");
In a #BeforeAll annotated method I elasticsearch url property like this
System.setProperty("spring.data.elasticsearch.cluster-nodes", container.getContainerIpAddress() + ":" + container.getMappedPort(9300));
From power shell when I check the running containers (during the test case debug pause), I find something like this under ports column : 0.0.0.0:32844->9200/tcp, 0.0.0.0:32843->9300/tcp
When I print container.getContainerIpAddress() + ":" + container.getMappedPort(9300), I get the same port mapped to 9300 in the container ports column, in this case localhost:32843, for sure the port is random and get changed in every new run.
when the code `conf = repo.save(conf); run, I get the following exception:
Caused by: org.apache.http.ProtocolException: Not a valid protocol version: This is not an HTTP port
at org.apache.http.impl.nio.codecs.AbstractMessageParser.parse(AbstractMessageParser.java:209)
at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:245)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.http.ParseException: Not a valid protocol version: This is not an HTTP port
at org.apache.http.message.BasicLineParser.parseProtocolVersion(BasicLineParser.java:148)
at org.apache.http.message.BasicLineParser.parseStatusLine(BasicLineParser.java:366)
at org.apache.http.impl.nio.codecs.DefaultHttpResponseParser.createMessage(DefaultHttpResponseParser.java:112)
at org.apache.http.impl.nio.codecs.DefaultHttpResponseParser.createMessage(DefaultHttpResponseParser.java:50)
at org.apache.http.impl.nio.codecs.AbstractMessageParser.parseHeadLine(AbstractMessageParser.java:156)
at org.apache.http.impl.nio.codecs.AbstractMessageParser.parse(AbstractMessageParser.java:207)
... 11 more
`
You are using a REST client to access Elasticsearch on port 9300. This is the port for the TransportClient. Wit a REST client you need to target port 9200.
Related
I have a testcontainer that creates a Oryd/Hydra container in a Junit 4 test.
#Bean
public GenericContainer hydra() {
WaitStrategy waitStrategy = Wait.forHttp("/health/ready").forStatusCode(200);
GenericContainer hydra =
new GenericContainer("oryd/hydra:1.4.8")
.withCommand("serve all --dangerous-force-http")
.withEnv("URLS_SELF_ISSUER", "http://127.0.0.1:4444/")
.withEnv("DSN", "memory")
.withEnv("SECRETS_SYSTEM", "youReallyNeedToChangeThis")
.withEnv("OIDC_SUBJECT_IDENTIFIERS_SUPPORTED_TYPES", "public,pairwise")
.withEnv("OIDC_SUBJECT_IDENTIFIERS_PAIRWISE_SALT", "youReallyNeedToChangeThis")
.withEnv("STRATEGIES_ACCESS_TOKEN", "jwt")
.withEnv("OIDC_SUBJECT_IDENTIFIERS_SUPPORTED_TYPES", "public")
.withEnv("URLS_CONSENT", "http://127.0.0.1:3000/consent")
.withEnv("URLS_LOGIN", "http://127.0.0.1:3000/login")
.withExposedPorts(4444, 4445)
.waitingFor(waitStrategy)
.withNetwork(network)
.withLogConsumer(consumer);
hydra.start();
return hydra;
}
The problem is with the environment variable "URLS_SELF_ISSUER". The clients of the Hydra server, validate that the URL of the server matches the value of "URLS_SELF_ISSUER". Its value should match the URL exposed to its clients, however testcontainers bind exposed port 4444 to a random port. So, URL will almost always be different from 127.0.0.1:4444.
This is a chicken and egg problem. I don't know what the port is until after the container starts, and then it's too late to update the variable.
Is there a way to know the exposed port so I can configure the container variable "URLS_SELF_ISSUER
" with the right URL??
I've tried several ports without success: 8787 is indeed busy serving rstudio. I could redirect rstudio, but shouldn't the following work?
from distributed import Client, LocalCluster
cluster = LocalCluster( scheduler_port = 8785 , n_workers = 2 )
Error:
/home/ec2-user/anaconda3/lib/python3.6/site-packages/distributed/bokeh/core.py:56: UserWarning:
Port 8787 is already in use.
Perhaps you already have a cluster running?
Hosting the diagnostics dashboard on a random port instead.
warnings.warn('\n' + msg)
tornado.application - ERROR - Multiple exceptions in yield list
...
~/anaconda3/lib/python3.6/multiprocessing/forkserver.py in connect_to_new_process(self, fds)
64 raise ValueError('too many fds')
65 with socket.socket(socket.AF_UNIX) as client:
---> 66 client.connect(self._forkserver_address)
67 parent_r, child_w = os.pipe()
68 child_r, parent_w = os.pipe()
ConnectionRefusedError: [Errno 111] Connection refused
I think you need to override the diagnostics_port instead:
https://github.com/dask/distributed/blob/2ec428ae5652f7d068baeb18223fb8a04ab8804e/distributed/deploy/local.py#L45-L49
I installed FreeRADIUS , Mysql inside docker Container
I exposed ports 1812 , 1813 , 3306 outside .
I imported Database to mysql .
I inserted this rows to databases
INSERT INTO nas VALUES (NULL , '0.0.0.0/0', 'myNAS', 'other', NULL , 'mysecret', NULL , NULL , 'RADIUS Client');
INSERT INTO radcheck (username, attribute, op, value) VALUES ('thisuser', 'User-Password', ':=', 'thispassword');
INSERT INTO radusergroup (username, groupname, priority) VALUES ('thisuser', 'thisgroup', '1');
INSERT INTO radgroupreply (groupname, attribute, op, value) VALUES ('thisgroup', 'Service-Type', ':=', 'Framed-User'), ('thisgroup', 'Framed-Protocol', ':=', 'PPP'), ('thisgroup', 'Framed-Compression', ':=', 'Van-Jacobsen-TCP-IP');
and i stopped freeradius ==> service freeradius stop
and iam using debug mode ==> freeradius -X
And when using this Command in another terminal for the same container ==> radtest thisuser thispassword 127.0.0.1 0 mysecret
Output: Server Accepted the request
But When the previous Command in another machine
Server does not see the request and output in the other machine is " No response "
Notes in the IN etc IN freeradius IN radiusd.conf file :
listen {
type = auth
ipaddr = *
port = 0 }
listen {
ipaddr = *
port = 0
type = acct }
How can i fix it ?
Adding the rows to the sql database is insufficient. You need to configure your sql instance in mods-available/sql to match your local database, uncomment read_clients in mods-available/sql, and list the sql module in the instantiate section in radiusd.conf to ensure it's loaded if it's not referenced elsewhere in one of the virtual servers.
After making these changes, restart the server. The SQL module should then read the clients list in on startup. Check the debug output freeradius -X to ensure the SQL module can connect to your database, and read the NAS entries in successfully.
The reason why your local connections work is because there's a client entry included for localhost in the clients.conf file that ships with the server.
I fixed this Issue by expose Ports in UDP Protocol -p 1813:1813/udp -p 1812:1812/udp
I setup a small project for learning the capabilities of Vert.x in a cluster environment but I'm facing some weird issues when I try to create the vertx instance inside a Docker image.
The project consists just in 2 verticles being deployed in different Docker containers and using the event bus to communicate with each other
If I use the vertx provided launcher:
Launcher.executeCommand("run", verticleClass, "--cluster")
(or just stating that the main class is io.vertx.core.Launcher and putting the right arguments)
Everything works both locally and inside docker images. But if I try to create the vertx instance manually with
Vertx.rxClusteredVertx(VertxOptions())
.flatMap { it.rxDeployVerticle(verticleClass) }
.subscribe()
Then it's not working in Docker (it works locally). Or, more visually
| | Local | Docker |
|:---------------: |:-----: |:------: |
| Vertx launcher | Y | Y |
| Custom launcher | Y | N |
By checking the Docker logs it seems that everything works. I can see that both verticles know each other:
Members [2] {
Member [172.18.0.2]:5701 - c5e9636d-b3cd-4e24-a8ce-e881218bf3ce
Member [172.18.0.3]:5701 - a09ce83d-e0b3-48eb-aad7-fbd818c389bc this
}
But when I try to send a message through the event bus the following exception is thrown:
WARNING: Connecting to server localhost:33845 failed
io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:33845
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:325)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Connection refused
... 11 more
Just for simplifying stuff I uploaded the project to Github. I tried to make it as simple as possible so it has 2 verticles & 2 main classes and lots of scripts for every combination
By checking the Docker logs it seems that everything works. I can see
that both verticles know each other
Yes, because your cluster manager works fine, but you should also make your event bus configuration be consistent in every node (machine/docker container) in your cluster because as mentioned in Vert.x documentation: cluster managers do not handle the event bus inter-node transport, this is done directly by Vert.x with TCP connections.
You have to set the cluster host on each node to be the IP address of this node itself and any arbitrary port number (unless you try to run more than Vert.x instance on the same node you have to choose a different port number for each Vert.x instance). The error you are facing is because the default cluster host is local host
For example if a node's IP address is 192.168.1.12 then you would do the following:
VertxOptions options = new VertxOptions()
.setClustered(true)
.setClusterHost("192.168.1.12") // node ip
.setClusterPort(17001) // any arbitrary port but make sure no other Vert.x instances using same port on the same node
.setClusterManager(clusterManager);
on another node whose IP address is 192.168.1.56 then you would do the following:
VertxOptions options = new VertxOptions()
.setClustered(true)
.setClusterHost("192.168.1.56") // other node ip
.setClusterPort(17001) // it is ok because this is a different node
.setClusterManager(clusterManager);
By researching a bit more into the Vert.X Launcher code I found out that internally it's doing more things than just "parsing" the inputs
In case vertx is configured to run in cluster mode the Launcher is setting the cluster address itself (via BareCommand class) so in order to replicate the Launcher behaviour with your own Main class (and having the flexibility to configure your vertx instance via code instead of via args) the code will be as follow:
fun main(args: Array<String>) {
Vertx.rxClusteredVertx(
VertxOptions(
clusterHost = getDefaultAddress()
)
)
.flatMap { it.rxDeployVerticle(verticleClass) }
.subscribe()
}
// As taken from io.vertx.core.impl.launcher.commands.BareCommand
fun getDefaultAddress(): String? {
val nets: Enumeration<NetworkInterface>
try {
nets = NetworkInterface.getNetworkInterfaces()
} catch (e: SocketException) {
return null
}
var netinf: NetworkInterface
while (nets.hasMoreElements()) {
netinf = nets.nextElement()
val addresses = netinf.inetAddresses
while (addresses.hasMoreElements()) {
val address = addresses.nextElement()
if (!address.isAnyLocalAddress && !address.isMulticastAddress
&& address !is Inet6Address) {
return address.hostAddress
}
}
}
return null
}
I've just installed a MarkLogic nosql database out of the box on a windows machine.
I wrote a simple javaclient to put data in to the database but I get this error:
org.apache.http.conn.HttpHostConnectException: Connection to http://my.caci.local:8003 refused
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:158)
The Marklogic database is started. This is the code :
DatabaseClient client = DatabaseClientFactory.newClient("localhost", 8003, "admin", "admin", Authentication.DIGEST);
XMLDocumentManager docMgr = client.newXMLDocumentManager(); BinaryDocumentManager binMgr = client.newBinaryDocumentManager();
DOMHandle handle = new DOMHandle(); for (int i = 0; i < AANT_PERSONEN; i++) {
Document document = createDocument(i);
String docId = "/zaak/" + 20;
handle.set(document);
docMgr.write(docId, handle); }
....
The Marklogic console reports the following ports to be active on my.caci.local:
Default :: Admin : 8001 [HTTP]
Default :: App-Services : 8000 [HTTP]
Default :: HealthCheck : 7997 [HTTP]
Default :: Manage : 8002 [HTTP]
I'm new to marklogic and this is my question:
- what port should I use to connect to from my java client?
In agreement with MystyxMac, I notice the console does not report a REST server on 8003.
Here's the documentation for setting up a REST server:
http://docs.marklogic.com/guide/rest-dev/intro#id_97899
You should also add users for the rest-reader, rest-writer, and rest-admin roles.
Hoping that helps,
Erik Hennum
For testing purposes you can simply switch the port you are using to 8000.
From the documentation:
When you install MarkLogic Server, a pre-configured REST API instance
is available on port 8000. This instance uses the Documents database
as the content database and the Modules database as the modules
database.
The instance on port 8000 is convenient for getting started, but you
will usually create a dedicated instance for production purposes.
http://docs.marklogic.com/guide/rest-dev/service#id_15309