I am trying to configure isync to fetch data from home.pl IMAP server. I have the simplest possible configuration file:
# ~/.mbsyncrc
IMAPStore store
Host host
Port 993
User kosciej#host.com.pl
PassCmd "pass poczta.home.pl/kosciej"
SSLType IMAPS
MaildirStore host-local
Path ~/.mail/host/
Channel host-inbox
Master :host-remote:"INBOX"
Slave :host-local:inbox
However it produces the error IMAP error: mailbox name inbox contains server's hierarchy delimiter. I connected to the server with openssl and found there is INBOX mailbox with . as delimiter.
a list "" *
* LIST (\HasNoChildren) "." "INBOX"
...
I managed to fix the problem with Pattern
Master :host-remote:
Slave :host-local:
Patterns inbox
But cannot understand what the error was about, it looks like "INBOX" has . somewhere?
Could anyone explain why the first version doesn't work?
Related
I'd like to preface this with the fact that I am completely new to SAP and SAP HANA, and OData.
I was tasked with pulling changes from a SAP HANA table and transfer those to Kafka.
I noticed there was a Kafka source connector already written, which can be found here.
For this task, I was given a URL, a username and a password.
The URL looks like this:
https://blablabla.companyName.com/companyName/Foo/Bar/Baz/Foo/Table/Resource.xsodata
And this is a sample of the source connector's configs:
# The first few settings are required for all connectors:
# a name, the connector class to run, and the maximum number of
# tasks to create.
name = odatav4-source-connector
connector.class = org.init.ohja.kafka.connect.odatav4.source.OData4SourceConnector
tasks.max = 1
# The remaining configs are specific to the OData v4 source connector.
# OData server host as either DNS or IP
sap.odata.host.address = services.odata.org
# OData server port
sap.odata.host.port = 443
# OData protocol (supported values are http or https)
sap.odata.host.protocol = https
# OData user name for basic authentication
# For services not requiring authentication this can be set to any value
sap.odata.user.name = anonymous
# OData user password for basic authentication
# For services not requiring authentication this can be set to any value
sap.odata.user.pwd = anonymous
# Optional list of service URL query parameters in the form of "param1=value1,param2=value2", e.g. sap-client=200
#sap.odata.query-params=
# none(default): DECIMALs will be mapped to Connect Decimal data type
# primitive: DECIMALs will be mapped to INT64(id scale = 0) anf FLOAT64
#sap.odata.decimal.mapping = none
# maximum amount of retries in case of service connection/communication errors (e.g. HTTP status codes 400-599)
#sap.odata.max.retries = 30
# The backoff strategy applied will select a random number of milliseconds
# to wait between min.retry.backoff.ms and max.retry.backoff.ms before starting
# the next retry.
#sap.odata.min.retry.backoff.ms = 20000
#sap.odata.max.retry.backoff.ms = 180000
# Timeout in milliseconds for establishing http connections
#sap.odata.connection.connect.timeout.ms=3000
# Timeout in milliseconds for reading data from a http connection
#sap.odata.connection.read.timeout.ms=10000
# Individual configurations for each OData v4 service entity.
# service and entityset build up the primary key for each OData configuration.
# OData v4 URL service path
sap.odata#00.service = /V4/Northwind/Northwind.svc/
# OData v4 entity set name
# The entity set name can be queried from the /$metadata service URL
sap.odata#00.entityset = Order_Details
# Kafka topic name the data for this OData service entity set will be pushed to
sap.odata#00.topic = Order_Details
# Execution interval in seconds for the scheduled data extractions
# Set to -1 to process subscription events only
#sap.odata#00.exec-period = 900
# If changes to entities selected by the first query should be tracked and returned as deltas in subsequent polls
# Set to 1 to enable odata delta mode
#sap.odata#00.track-changes = 0
# Paging mode (server or client) determines the type of paging
# server: use HTTP prefer-headers to request a maximum package size from the odata server
# client: use query functions skip and top (not compatible to change tracking)
#sap.odata#00.paging.mode = server
# Packaging size in count of entity set records
#sap.odata#00.paging.size = 50000
# Optional: Hierarchy level up to which recommendations for the expand.list configuration (query option $expand) will
# be shown in the Confluent Control Center
#sap.odata#00.expand.level = 1
# Optional: List of expand query options that will define the deep structure of returned entity messages
#sap.odata#00.expand.list =
# Optional: comma separated list of selected non-key fields to be extracted
#sap.odata#00.projection =
# Optional: filter query options
# Supported logical operations/options are: eq, ne, le, lt, ge, gt, bt, nb, in
#sap.odata#00.select#00.fieldname =
#sap.odata#00.select#00.option =
#sap.odata#00.select#00.low =
#sap.odata#00.select#00.high =
# If set to 1 the connector will subscribe to push-notifications issued by the corresponding OData service entity
#sap.odata#00.subscription.enable = 0
So I tried to create my own, like so:
{
"name": "sap-hana-source-connector",
"config": {
"connector.class": "org.init.ohja.kafka.connect.odatav4.source.OData4SourceConnector",
"sap.odata.user.name": "username",
"sap.odata.host.address": "blablabla.companyName.com",
"sap.odata.host.port": "443",
"sap.odata.host.protocol": "https",
"sap.odata#00.service": "/companyName/Foo/Bar/Baz/Foo/Table/Resource.xsodata",
"sap.odata#00.entityset": "Resource",
"sap.odata.user.pwd": "pwd"
}
}
The issue is that the only error I get is this:
{
"error_code": 400,
"message": "Connector configuration is invalid and contains the following 14 error(s):\nInvalid configuration sap.odata.host.address: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.host.protocol: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.host.port: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.user.name: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.user.pwd: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.max.retries: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.min.retry.backoff.ms: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.max.retry.backoff.ms: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.connection.connect.timeout.ms: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.connection.read.timeout.ms: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.query-params: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.trace.mode: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.trace.path: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.decimal.mapping: No configured service reachable. Maybe invalid destination configuration?\nYou can also find the above list of errors at the endpoint `/connector-plugins/{connectorType}/config/validate`"
}
As someone who is completely new to OData and SAP, I don't know how I'd debug this.
I noticed that, in the OData's Kafka Source Connector documentation the services end in .svc and not .xsodata; so maybe it's something to do with that?
Also, what am I supposed to be for the sap.odata#00.entityset config?
Is there a way to get a more detailed error message?
Thanks.
I have been scouring the internet with no luck. I have a basic LUA script for HAProxy, which looks like this:
core.Info("LUA script for parsing request ID element - loaded");
function parseId(txn, salt)
local payload = txn.sf:req_body()
-- parses hex value from element named "ID". Example payload: {"Platform":"xyz.hu","RecipientId":"xyz.hu","Channel":"xyz","CallbackURL":"http://x.x.x.x:123","ID":"5f99453d000000000a0c5164233e0002"}
local value = string.sub(string.match(payload, "\"ID\":\"[0-9a-f]+\""), 8, -2)
core.Info("ID : " .. value)
return value
end
-- register HAProxy "fetch"
core.register_fetches("parseId", parseId)
What it does is what it says: takes a 32 characater long ID from an incoming request. In the HAProxy config file, the result is used for sticky-session handling:
stick-table type string len 32 size 30k expire 30m
stick on "lua.parseId" table gw_back
This produces two lines of log for each request:
ID: xyz which is logged from the LUA script
The detailed request data which is logged from the HAProxy config file using "log-format", e.g.:
Jan 20 22:13:52 localhost haproxy[12991]: Client IP:port = [x.x.x.x:123], Start Time = [20/Jan/2022:22:13:52.069], Frontend Name = [gw_front], Backend Name = [gw_back], Backend Server = [gw1], Time to receive full request = [0 ms], Response time = [449 ms], Status Code = [200], Bytes Read = [308], Request = ["POST /Gateway/init HTTP/1.1"], ID = [""], Request Body = [{"Platform":"xyz.hu","RecipientId":"xyz.hu","Channel":"xyz","CallbackURL":"http://x.x.x.x:123","ID":"61e9d03e000000000a0c5164233e0002"}]
I wanted to extend logging due to some strange issues happening sometimes, so I wanted to one (or both) of below approaches:
Pass the "ID" value back from the LUA script into the HAProxy config as a variable, and log it along with the request details. I can log the full request body, but don't want to due to GDPR and whatnot.
Get some request details in the LUA script itself, and log it along with the ID.
So, basically, to be able to connect the ID with the request details. If multiple requests are coming to same URL very quickly, it is difficult to find which of them belongs to a specific ID. However I couldn't accomplish these.
For the first one, I added this line into the LUA before returning the "value" variable:
txn:set_var("req_id", value)
I was hoping this would create a variable in HAProxy called "req_id", and I can log it with "log-format", but all I got was empty string:
ID = [""]
For the second one, I'm at a complete loss. I'm not able to find ANY documentation on these. I have been scouring the internet with no luck. E.g. the txn.sf:req_body() function which I know is working, I simply cannot find it documented anywhere, so I'm not sure what other functions are available to get some request details.
Any ideas for either or both of my approaches? I'm attaching my full HAProxy config here at the end, just in case:
global
log 127.0.0.1 len 10000 local2 debug
chroot /var/lib/haproxy
user haproxy
group haproxy
daemon
lua-load /opt/LUA/parseId.lua
stats socket /etc/haproxy/haproxysock level admin
defaults
log global
option httplog
option dontlognull
mode http
timeout connect 5000
timeout client 50000
timeout server 50000
# Request body is temporarily logged in test environment
log-format "Client IP:port = [%ci:%cp], Start Time = [%tr], Frontend Name = [%ft], Backend Name = [%b], Backend Server = [%s], Time to receive full request = [%TR ms], Response time = [%Tr ms], Status Code = [%ST], Bytes Read = [%B], Request = [%{+Q}r], ID = [%{+Q}[var(txn.req_id)]], Request Body = [%[capture.req.hdr(0)]]"
frontend gw_front
bind *:8776
option http-buffer-request
declare capture request len 40000
http-request capture req.body id 0
http-request track-sc0 src table gw_back
use_backend gw_back
backend gw_back
balance roundrobin
stick-table type string len 32 size 30k expire 30m
stick on "lua.parseId" table gw_back
# Use HTTP check mode with /ping interface instead of TCP-only check
option httpchk POST /Gateway/ping
server gw1 x.x.x.x:8080 check inter 10s
server gw2 y.y.y.y:8080 check inter 10s
listen stats
bind *:8774 ssl crt /etc/haproxy/haproxy.cer
mode http
maxconn 5
stats enable
stats refresh 10s
stats realm Haproxy\ Statistics
stats uri /stats
stats auth user:password
Need to post a byte message to solace queue using Jmeter. I have tried in following manner might be am incorrect but tried with following:
Use JMSPublisher sampler
create jndi.properties file and put in jmeter/lib
jndi.properties
java.naming.factory.initial = com.solacesystems.jndi.SolJNDIInitialContextFactory
java.naming.provider.url = smf://<remote IP and port>
java.naming.security.principal=<username>
java.naming.security.credentials=<password>
Solace_JMS_VPN=<VPN Name>
in JMSPublisher sampler (in GUI)
Connection Factory = connectionFactory
Destination = (Queue Name )
Message Type (radio button---Byte message)
Content encoding -- RAW
in text area ---> (Byte message)
Note : I have used actual values of IP/port/username/port/queuename/bytemessage, cannot share those. Soljms jar is available in lib folder too.
getting error :
Response message: javax.naming.NamingException: JNDI lookup failed - 503: Service Unavailable [Root exception is (null) com.solacesystems.jcsmp.JCSMPErrorResponseException: 503: Service Unavailable]
Though it is working perfectly fine when did with java spring boot. There used properties files in place of JNDI.
It would be great if anyone can guide me , please do not give activeMQ JNDI am actively looking for posting on solace queue or create connection to solace appliances through Jmeter.
I don't think you should be putting your Byte message into the textarea as it accepts either plain text or an XStream object, consider providing your payload via binary file(s) instead
If you're capable of sending the message using Java code you should be able to replicate the same using:
JMeter's JSR223 Sampler with Groovy language (Java syntax will work)
Or JUnit Request sampler if you need "strict" java
Maybe (hopefully) I'm missing something very simple, but I can't seem to figure this out.
I have a set of gRPC services that I would like to put behind a nghttpx proxy. For this I need to be able to configure my client with a channel on a non-root url. Eg.
channel = grpc.insecure_channel('localhost:50051/myapp')
stub = MyAppStub(channel)
This wasn't working immediately through the proxy (it just hangs), so I tested with a server on the sub context.
server = grpc.server(executor)
service_pb2.add_MyAppServicer_to_server(
MyAppService(), server)
server.add_insecure_port('{}:{}/myapp'.format(hostname, port))
server.start()
I get the following
E1103 21:00:13.880474000 140735277326336 server_chttp2.c:159]
{"created":"#1478203213.880457000","description":"OS Error",
"errno":8,"file":"src/core/lib/iomgr/resolve_address_posix.c",
"file_line":115,"os_error":"nodename nor servname provided, or not known",
"syscall":"getaddrinfo","target_address":"[::]:50051/myapp"}
So the question is - is it possible to create gRPC channels on non-root urls?
As confirmed here, this is not possible. I will route traffic via subdomains in nghttpx.
i want to check my server connection to know if its available or not to inform the user..
so how to send a pkg or msg to the server (it's not SQL server; it's a server contains some serviecs) ...
thnx in adcvance ..
With all the possibilities for firewalls blocking ICMP packets or specific ports, the only way to guarantee that a service is running is to do something that uses that service.
For instance, if it were a JDBC server, you could execute a non-destructive SQL query, such as select * from sysibm.sysdummy1 for DB2. If it's a HTTP server, you could create a GET packet for index.htm.
If you actually have control over the service, it's a simple matter to create a special sub-service to handle these requests (such as you send through a CHECK packet and get back an OKAY response).
That way, you avoid all the possible firewall issues and the test is a true end-to-end one. PINGs and traceroutes will be able to tell if you can get to the machine (firewalls permitting) but they won't tell you if your service is functioning.
Take this from someone who's had to battle the network gods in a corporate environment where machines are locked up as tight as the proverbial fishes ...
If you can open a port but don't want to use ping (i dont know why but hey) you could use something like this:
import socket
host = ''
port = 55555
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((host, port))
s.listen(1)
while 1:
try:
clientsock, clientaddr = s.accept()
clientsock.sendall('alive')
clientsock.close()
except:
pass
which is nothing more then a simple python socket server listening on 55555 and returning alive