I have a docker-compose file with several containers in, two of which are supposed to communicate via a Redis DB. Both containers have connections to Reids and I can read/write from both. However I'd like a container to be triggered every time the something is added from the other container. I thought I could accomplish this via Redis Sub/Pub but it when I run the code it never triggers anything, even when I can see I've added new items to the Redis queue.
From this I have two questions:
1. Is what I'm looking to do even possible? Can I do publish/subscribe in in two separate docker containers and expect it work as described above?
2. If it is possible, can someone please point me below where I"m going wrong with this tools?
This is my function that I add new data to the Redis queue and then publish the data in Docker container 1.
func redisShare(key string, value string) {
jobsQueue.Set(key, value, 0) //setting in the queue
jobsQueue.Publish(key, value) //publishing for the other docker container to notice
fmt.Println("added ", key, "with a value of ", value, "to the redis queue")
}
I'm using this line in my other docker container to subscribe to the Redis queue and listen for changes:
redisdb.Subscribe()
I would expect if something was added to the redis queue it would share the data to the other container and I'd see the message received, but right now Docker Container 2 just runs and then closes.
Thanks!
Just in case anyone else wonders about the answer: I ended up using a combination of both Aleksandrs and sui's answers.
In my first Docker container I published the results to a specific channel:
publishData := redisdb.Subscribe("CHANNELNAME")
And then in my second Docker container that was subscribing to the channel, thanks to sui for the assistance on this part, I subscribed to the channel and pulled the both the UID and the IP information like so:
ch := pubsub.Channel()
for msg := range ch {
fmt.Println(msg.Payload)
s := strings.Split(msg.Payload, ":")
UID, IP := s[0], s[1]
fmt.Println(UID, IP)
}
This is working great for me so far - thanks so much to sui and Aleksandrs for the assistance!
As from docs
receiver.go
package main
import (
"fmt"
"github.com/go-redis/redis"
)
func main() {
c := redis.NewClient(&redis.Options{
Addr: ":6379",
})
pubsub := c.Subscribe("mychannel1")
// Wait for confirmation that subscription is created before publishing anything.
_, err := pubsub.Receive()
if err != nil {
panic(err)
}
// Go channel which receives messages.
ch := pubsub.Channel()
// Consume messages.
for msg := range ch {
fmt.Println(msg.Channel, msg.Payload)
}
}
sender.go
package main
import (
"time"
"github.com/go-redis/redis"
)
func main() {
c := redis.NewClient(&redis.Options{
Addr: ":6379",
})
// Publish a message.
for range time.Tick(time.Second) {
err := c.Publish("mychannel1", "hello").Err()
if err != nil {
panic(err)
}
}
}
Related
I am developing a go CUI for docker container. I need container ID for only selected container Name.
I tried it this way:
id, err := exec.Command("/bin/sh", "-c", "docker", fmt.Sprintf("ps --no-trunc -aqf name=%s"),conName).Output()
if err != nil {
log.Fatal(err)
}
fmt.Println("CONTAINER ID:", id)
Output:
CONTAINER ID: []
this worked for me (may be you don't need sudo):
id, err := exec.Command("sudo", "docker", "ps", "--no-trunc", "-aqf", fmt.Sprintf("name=%s",conName)).Output()
if err != nil {
log.Fatal(err)
}
fmt.Println("CONTAINER ID:", string(id))
Your mistake was, that conName was not inside the parentheses and you also have to convert id to a string as exec.Command returns you a byte array. Also all arguments should be listed separately, if you place several arguments separated by whitespaces as one string, it won't work.
I'm trying to resolve mx records in a kubernetes pod.
The dnsjava library works when tested on mac and ubuntu outside of a container but returns an empty array once deployed.
What needs to be available in k8s or the docker image for this to work?
See https://github.com/dnsjava/dnsjava
EDIT 1
Record[] records;
try {
records = new Lookup(mailDomain, Type.MX).run();
} catch (TextParseException e) {
throw new IllegalStateException(e);
}
if (records != null && records.length > 0) {
for (final Record record : records) {
MXRecord mx = (MXRecord) record;
//do something with mx...
}
} else {
log.warn("Failed to determine MX record for {}", mailDomain);
}
The log.warn is always executed in K8s. The docker image is openjdk:11-jdk-slim i.e. it's Debian. I just tested on Debian outside of Docker and it worked as well.
In the end I couldn't get dnsjava to work in docker/k8s.
I used JNDI directly, following https://stackoverflow.com/a/16448180/400048
this works without any issues exactly as given in that answer.
Our application is an ASP.NET Core 2.0 WebAPI deployed in Linux Docker containers and running in Kubernetes.
During load testing, we discovered intermittent spikes in CPU usage that our application would never recover from.
We used perfcollect to collect traces from a container so that we could compare a successful test and a test with CPU spikes. We discovered that around 75% of the CPU time in the failing test was spent in JIT_MonRelaibleEnter_Protable, an interface of lock operations. The caller was System.Diagnostics.TraceSource.dll.
Our application was ported from .NET Framework and contained a lot of calls to System.Diagnostics.Trace.WriteLine(). When we removed all of these, our CPU/memory usage reduced by more than 50% and we don't see the CPU spikes anymore.
I want to understand the cause of this issue.
In the corefx repo, I can see that a default trace listener is setup in TraceInternal.cs:
public static TraceListenerCollection Listeners
{
get
{
InitializeSettings();
if (s_listeners == null)
{
lock (critSec)
{
if (s_listeners == null)
{
// In the absence of config support, the listeners by default add
// DefaultTraceListener to the listener collection.
s_listeners = new TraceListenerCollection();
TraceListener defaultListener = new DefaultTraceListener();
defaultListener.IndentLevel = t_indentLevel;
defaultListener.IndentSize = s_indentSize;
s_listeners.Add(defaultListener);
}
}
}
return s_listeners;
}
}
I can see that DefaultTraceListener.cs calls Debug.Write():
private void Write(string message, bool useLogFile)
{
if (NeedIndent)
WriteIndent();
// really huge messages mess up both VS and dbmon, so we chop it up into
// reasonable chunks if it's too big
if (message == null || message.Length <= InternalWriteSize)
{
Debug.Write(message);
}
else
{
int offset;
for (offset = 0; offset < message.Length - InternalWriteSize; offset += InternalWriteSize)
{
Debug.Write(message.Substring(offset, InternalWriteSize));
}
Debug.Write(message.Substring(offset));
}
if (useLogFile && !string.IsNullOrEmpty(LogFileName))
WriteToLogFile(message);
}
In Debug.Unix.cs, I can see that there is a call to SysLog:
private static void WriteToDebugger(string message)
{
if (Debugger.IsLogging())
{
Debugger.Log(0, null, message);
}
else
{
Interop.Sys.SysLog(Interop.Sys.SysLogPriority.LOG_USER | Interop.Sys.SysLogPriority.LOG_DEBUG, "%s", message);
}
}
I don't have a lot of experience working with Linux but I believe that I can simulate the call to SysLog by running the following command in the container:
logger --socket-errors=on 'SysLog test'
When I run that command, I get the following response:
socket /dev/log: No such file or directory
So it looks like I can't successfully make calls to SysLog from the container. If this is indeed what is going on when I call Trace.WriteLine(), why is it causing locking issues in my application?
As far as I can tell, EnvVar_DebugWriteToStdErr is not set in my container so it should not be attempting to write to StdErr.
The solution can be that rsyslog is not running. Is that installed in your container? Use a base image that has rsyslog built in.
This link can help too.
I'm looking to have a static landing page at url="/" and then have any files url="/"+file be served using a template.
I have the template working fine with this code
package main
import (
"html/template"
"log"
"net/http"
"os"
"path"
)
func main() {
fs := http.FileServer(http.Dir("static"))
http.Handle("/static/", http.StripPrefix("/static/", fs))
http.HandleFunc("/", serveTemplate)
log.Println("Listening...")
http.ListenAndServe(":5000", nil)
}
func serveTemplate(w http.ResponseWriter, r *http.Request) {
lp := path.Join("templates", "layout.html")
fp := path.Join("templates", r.URL.Path)
// Return a 404 if the template doesn't exist
info, err := os.Stat(fp)
if err != nil {
if os.IsNotExist(err) {
http.NotFound(w, r)
return
}
}
// Return a 404 if the request is for a directory
if info.IsDir() {
http.NotFound(w, r)
return
}
templates, err := template.ParseFiles(lp, fp)
if err != nil {
log.Print(err)
http.Error(w, "500 Internal Server Error", 500)
return
}
templates.ExecuteTemplate(w, "layout", nil)
}
So this works fine. Basically, I think I need to do two things. One, add another http.Handle or http.HandlerFunc in my main() function which handles a single html file, and then have my error checkers to redirect there instead of throwing a 404 error.
Please help how I may do this or provide a better solution?
I'd suggest reading through: http://golang.org/doc/articles/wiki/#tmp_6 - it covers much of this.
Specifically:
You are blocking on every request to read the file system (bad!);
You're then parsing your template files (slow) on every request;
Using a part of the URL path to directly read from the filesystem is a huge security problem (even if Go tries to escape it, expect someone to beat it). Be very careful about this
You should also parse your templates during program startup (i.e. at the start of your main()) once only. Read all the templates in ahead of time from a dir using tmpl := template.Must(template.ParseGlob("/dir")) - which will allow you to look-up your templates from your route. The html/template docs cover this well.
Note that you'll need to write some logic to catch when a template you are trying to match from the route does not exist in your handler.
I'd also look at using gorilla/mux if you want a few more features. You could write a not found handler that re-directs to / with a 302 (temp. re-direct) instead of raising a 404.
r := mux.NewRouter()
r.HandleFunc("/:name", nameHandler)
r.HandleFunc("/", rootHandler)
r.NotFoundHandler(redirectToRoot)
http.Handle("/", r)
log.Fatal(http.ListenAndServe(":8000", nil))
func redirectToRoot(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/", http.StatusSeeOther)
}
Hope that helps.
I have an InstallScript project in IS2010. It has a handful of services that get installed. Some are C++ exes and use the "InstallShield Object for NT Services". Others are Java apps installed as services with Java Service Wrapper through LaunchAppAndWait command line calls. Tomcat is also being installed as a service through a call to its service.bat.
When the installer runs in upgrade mode, the services are reinstalled, and the settings (auto vs. manual startup, restart on fail, log-on account, etc.) are reverted to the defaults.
I would like to save the service settings before the file transfer and then repopulate them afterward, but I haven't been able to find a good mechanism to do this. How can I save and restore the service settings?
I got this working by reading the service information from the registry in OnUpdateUIBefore, storing it in a global variable, and writing the information back to the registry in OnUpdateUIAfter.
Code:
export prototype void LoadServiceSettings();
function void LoadServiceSettings()
number i, nResult;
string sServiceNameArray(11), sRegKey, sTemp;
BOOL bEntryFound;
begin
PopulateServiceNameList(sServiceNameArray);
RegDBSetDefaultRoot(HKEY_LOCAL_MACHINE);
//write service start values to the registry
for i = 0 to 10
if (ServiceExistsService(sServiceNameArray(i))) then
sRegKey = "SYSTEM\\CurrentControlSet\\Services\\" + sServiceNameArray(i);
nResult = RegDBSetKeyValueEx(sRegKey, "Start", REGDB_NUMBER, sServiceSettings(i), -1);
if(nResult < 0) then
MessageBox ("Unable to save service settings: " + sServiceNameArray(i) + ".", SEVERE);
endif;
endif;
endfor;
RegDBSetDefaultRoot(HKEY_CLASSES_ROOT); //set back to default
end;
export prototype void SaveServiceSettings();
function void SaveServiceSettings()
number i, nType, nSize, nResult;
string sServiceNameArray(11), sRegKey, sKeyValue;
begin
PopulateServiceNameList(sServiceNameArray);
RegDBSetDefaultRoot(HKEY_LOCAL_MACHINE);
for i = 0 to 10
if (ServiceExistsService(sServiceNameArray(i))) then
//get service start values from registry
sRegKey = "SYSTEM\\CurrentControlSet\\Services\\" + sServiceNameArray(i);
nResult = RegDBGetKeyValueEx(sRegKey, "Start", nType, sKeyValue, nSize);
if(nResult < 0) then
MessageBox ("Unable to save service settings: " + sServiceNameArray(i) + ".", SEVERE);
endif;
sServiceSettings(i) = sKeyValue;
endif;
endfor;
RegDBSetDefaultRoot(HKEY_CLASSES_ROOT); //set back to default
end;