When running the code below, the program starts at around 1.5M and then gradually grows until 6.4M. I'm wondering why.
Removing time.sleep fixes the issue.
Is there a way to use the for-select pattern with a default and sleep some time in the default without any mem change?
Calling runtime.GC() after the sleep does fix the issue. Can we achieve the same thing without having to call the GC ?
package main
import (
"time"
)
func main() {
c := make(chan struct{})
for {
select {
case <-c:
//some work
default:
//some work
time.Sleep(time.Millisecond * 1)
}
}
}
Same with :
package main
import (
"time"
)
func main() {
c := make(chan struct{})
for {
select {
case <-c:
case <-time.After(time.Millisecond * 10):
}
}
}
After some time researching, I achieved it with the following code. Still wonder why time.sleep increases mem usage?
package main
import (
"time"
)
func main() {
c := make(chan bool)
timer := time.NewTimer(0)
for {
select {
case <-c:
default:
timer.Reset(time.Millisecond * 1)
<-timer.C
}
}
}
You can implement a timeout using a select and receive from <-time.After():
select {
case res := <-c:
fmt.Println("do some work")
case <-time.After(time.Second * 1):
fmt.Println("timeout")
}
In case you want to understand how your program is utilizing memory you can do profiling. Here is a nice article about this topic.
After some time researching, I achieved it with the following code.
package main
import (
"time"
)
func main() {
c := make(chan bool)
timer := time.NewTimer(0)
for {
select {
case <-c:
default:
timer.Reset(time.Millisecond * 1)
<-timer.C
}
}
}
Related
I am trying to code a Docker Monitoring software in Golang.
my Code looks as followed:
package main
import (
"bytes"
"context"
"fmt"
"time"
"github.com/docker/docker/api/types"
"github.com/docker/docker/client"
)
func main() {
ctx := context.Background()
cli, err := client.NewClientWithOpts(client.FromEnv)
if err != nil {
panic(err)
}
containers, err := cli.ContainerList(ctx, types.ContainerListOptions{})
if err != nil {
panic(err)
}
for _, container := range containers {
out, err := cli.ContainerLogs(ctx, container.ID, types.ContainerLogsOptions{
ShowStderr: true,
ShowStdout: true,
Timestamps: false,
Follow: true,
Tail: "40"})
if err != nil {
panic(err)
}
fmt.Println("The \"" + container.Image + "\" container, with the ID \"" + container.ID + "\" logged: ")
fmt.Println()
buf := new(bytes.Buffer)
fmt.Println(buf.ReadFrom(out))
fmt.Println(buf.String())
}
time.Sleep(time.Second * 3)
}
The problem is that the execution of the above code stops on the fmt.Println(buf.ReadFrom(out)) statement. The code used to work, but it suddenly just doesn't anymore. Either it stops without an error, or it returns an empty String.
The client I am trying to collect the logs from is also coded by myself, and it looks like follows:
package main
import (
"log"
"time"
)
func main() {
for i := 0; i > -1; i++ {
log.Output(1, "Hello World logged!")
time.Sleep(time.Minute)
}
}
I already tried debugging and checking Variables, but I just can't get to the source of the Problem.
I am really not sure as I don't have any error logs to confirm my hypothesis.
But could it be the case that as the ContainerLogs returns a stream (io.ReadCloser), maybe the stream itself hasn't closed yet?
If this is a possibility, you can either do a dry run first to test out this theory by adding a timeout and logging it after every small duration ?
one possible way to do this is
select {
case <-time.After(5 * time.Second):
fmt.Println("Timeout exceeded while reading container logs")
case <-ctx.Done():
fmt.Println("Context cancelled while reading container logs")
case b := <-out:
if b != nil {
buf.Write(b)
}
}
I am using text/scanner package to parse some arbitrary expressions. I am currently trying to implement a not in option, that is, if the current identifier is not, and the next is in, parse it using function notin(left, right), and otherwise we parse it as negate(right).
I've essentially got the code to manage these cases however, I am unable to rewind the scanner in case the next token is not in. I've tried by recording the position and then reassigning it later, but to no avail and haven't been able to find a different solution.
func readToken(stream *scanner.Scanner) {
switch stream.Scan() {
case scanner.Ident:
switch stream.TokenText() {
case "in":
in(left, right)
case "not":
oldPosition := stream.Position
nextToken := stream.Scan()
if nextToken == scanner.Ident {
switch stream.TokenText() {
case "in":
fmt.Println("notin")
default:
// how do we rewind the scanner?
stream.Position = oldPosition
fmt.Println("negate default")
}
} else {
fmt.Println("negate no-ident")
}
}
}
}
How can I rewind the scanner when I don't find a valid identifier?
Edit, I also tried using Peek() as below, but that still changes the state to the point that I'd need to rewind as well.
// other code
case "not":
nextIdent, err := getNextIdent(stream)
if err != nil {
fmt.Println("negate no-ident")
} else {
switch nextIdent {
case "in":
fmt.Println("notin")
default:
fmt.Println("negate default")
}
}
// other code
func getNextIdent(s *scanner.Scanner) (string, error) {
var nextIdent string
ch := s.Peek()
// skip white space
for s.Whitespace&(1<<uint(ch)) != 0 {
ch = s.Next()
}
if isIdentRune(ch, 0) {
nextIdent = string(ch)
ch = s.Next()
nextIdent += string(ch)
for i := 1; isIdentRune(ch, i); i++ {
ch = s.Next()
if s.Whitespace&(1<<uint(ch)) != 0 {
break
}
nextIdent += string(ch)
}
return nextIdent, nil
}
return "",errors.New("not a ident")
}
Note, the code I've got is a forked from Knetic/govaluate combined with a PR from GH user generikvault and some other forks. The full code can be found on my Github profile
By looking at the API references of text/scanner, I can't seem to find a way to rewind the scanner the way you want.
However, the Peek() method would get you the next rune without advancing the scanner. Inside the "not" case, you can use it to peek in advance to see if it matches.
I'm still getting started with ReactiveCocoa and functional reactive programming concepts, so maybe this is a dumb question.
ReactiveCocoa seem naturally designed to react to streams of live data, touch events or accelerometer sensor input etc.
Is it possible to apply finite impulse response filters in ReactiveCocoa in an easy, reactive fashion? Or if not, what would be the least-ugly hacky way of doing this? How would one go about implementing something like a simple moving average?
Ideally looking for an Swift 2 + RA4 solution but also interested in if this is possible at all in Objective C and RA2/RA3.
What you actually need is a some sort of period buffer, which will keep a period of values buffered and only start sending out when the buffer has reached capacity (the code below is heavenly inspired on takeLast operator)
extension SignalType {
func periodBuffer(period:Int) -> Signal<[Value], Error> {
return Signal { observer in
var buffer: [Value] = []
buffer.reserveCapacity(period)
return self.observe { event in
switch event {
case let .Next(value):
// To avoid exceeding the reserved capacity of the buffer, we remove then add.
// Remove elements until we have room to add one more.
while (buffer.count + 1) > period {
buffer.removeAtIndex(0)
}
buffer.append(value)
if buffer.count == period {
observer.sendNext(buffer)
}
case let .Failed(error):
observer.sendFailed(error)
case .Completed:
observer.sendCompleted()
case .Interrupted:
observer.sendInterrupted()
}
}
}
}
}
based on that you can map it to any algorithm you want
let pipe = Signal<Int,NoError>.pipe()
pipe.0
.periodBuffer(3)
.map { Double($0.reduce(0, combine: +))/Double($0.count) } // simple moving average
.observeNext { print($0) }
pipe.1.sendNext(10) // does nothing
pipe.1.sendNext(11) // does nothing
pipe.1.sendNext(15) // prints 12
pipe.1.sendNext(7) // prints 11
pipe.1.sendNext(9) // prints 10.3333
pipe.1.sendNext(6) // prints 7.3333
Probably the scan signal operator is what you're looking for. Inspired by Andy Jacobs' answer, I came up with something like this (a simple moving average implementation):
let (signal, observer) = Signal<Int,NoError>.pipe()
let maxSamples = 3
let movingAverage = signal.scan( [Int]() ) { (previousSamples, nextValue) in
let samples : [Int] = previousSamples.count < maxSamples ? previousSamples : Array(previousSamples.dropFirst())
return samples + [nextValue]
}
.filter { $0.count >= maxSamples }
.map { $0.average }
movingAverage.observeNext { (next) -> () in
print("Next: \(next)")
}
observer.sendNext(1)
observer.sendNext(2)
observer.sendNext(3)
observer.sendNext(4)
observer.sendNext(42)
Note: I had to move average method into a protocol extension, otherwise the compiler would complain that the expression was too complex. I used a nice solution from this answer:
extension Array where Element: IntegerType {
var total: Element {
guard !isEmpty else { return 0 }
return reduce(0){$0 + $1}
}
var average: Double {
guard let total = total as? Int where !isEmpty else { return 0 }
return Double(total)/Double(count)
}
}
I have a sequence of asynchronous methods defined like so:
func step1(input: Step1InputData, completion: (Step1OutputData -> Void)) { /* do something */ }
func step2(input: Step1OutputData, completion: (Step2OutputData -> Void)) { /* do something */ }
// etc...
As you can see, the output of step1 is the input of step2. These types all implement the StepData protocol:
protocol StepData {}
class Step1InputData : StepData { }
class Step1OutputData : StepData { }
class Step2OutputData : StepData { }
Finally, I have this custom operator:
infix operator => { associativity left }
func => <P:StepData, Q:StepData, R:StepData> (left:((P, Q -> Void) -> Void), right:((Q, R -> Void) -> Void)) -> ((P, R -> Void) -> Void) {
return { (x, completion) in
left(x, { y in
right(y, completion)
})
}
}
... which means I can write the following:
let combinedStep = step1 => step2
Which is great because it's super readable, and crucially it enforces type safety between the steps.
The problem is that I want to be able to persist the progress of a combinedStep. For example, step1 could be an image upload and step2 could be a local storage write. If step2 fails, the next time I try, I want to pick up where I left off, rather than re-uploading the image. Of course, there could be any number of steps chained together.
I could come up with a system that has an array of steps and manages the passing of data between steps and would be able to persist the progress, however, I can't think of a way to do this while still keeping the compile-time type safety.
Can someone with more experience of functional programming point me in the right direction?
I would suggest you look into frameworks like PromiseKit, BrightFutures, and ReactiveCocoa.
Those framework are in the same problem and solution space as you, asynchronous operations that need to be composed, and they all provide multiple ways to handle errors and retries.
If you like any of those you could adopt them in place of your custom implementation, and leverage the support of a big community of developers. Or you could just find inspiration from their code to bring back in your.
Enjoy
I think you need to make your operator right associative and pass an enum in the completion handler to allow for failure.
enum StepResult<Output> {
case Success(Output)
case Failure((StepResult<Output>->Void)->Void)
static func toFailHandlerWithInput<Input>(input: Input, processor: (Input,(StepResult<Output>)->Void)->Void) -> StepResult<Output> {
let handler : ((StepResult<Output>->Void)->Void) = { completion in
processor(input) { pout in
completion(pout)
}
}
return .Failure(handler)
}
}
infix operator => { associativity right }
func =><Input,Intermediate,Output>(left: (Input,(StepResult<Intermediate>)->Void)->Void, right: (Intermediate,(StepResult<Output>)->Void)->Void) -> (Input,(StepResult<Output>)->Void)->Void {
var mergedOp : ((Input,(StepResult<Output>)->Void)->Void)!
mergedOp = { input,completion in
left(input) { intermediate in
switch intermediate {
case .Success(let output):
right(output, completion)
case .Failure:
let failure = StepResult.toFailHandlerWithInput(input, processor: mergedOp)
completion(failure)
}
}
}
return mergedOp
}
var counter1 = 0
func step1(input: Int, completion: (StepResult<Double>)->Void) {
print("performing step 1...")
counter1 += 1
if ( counter1 > 1 ) {
print("Step 1 will succeed...")
let result = 2.0 * Double(input + counter1)
completion(.Success(result))
} else {
print("Step 1 fails...")
completion(StepResult.toFailHandlerWithInput(input, processor: step1))
}
}
var counter2 = 0
func step2(input: Double, completion: (StepResult<Double>)->Void) {
print("performing Step 2...")
counter2 += 1
if ( counter2 > 2 ) {
print("Step 2 will succeed...")
let result = 3 * input + Double(counter2)
completion(.Success(result))
} else {
print("Step 2 fails...")
completion(StepResult.toFailHandlerWithInput(input, processor: step2))
}
}
var counter3 = 0
func step3(input: Double, completion: (StepResult<Double>)->Void) {
print("performing Step 3...")
counter3 += 1
if ( counter3 > 1 ) {
print("Step 3 will succeed...")
let result = 4 * input + Double(counter3)
completion(.Success(result))
} else {
print("Step 3 fails...")
completion(StepResult.toFailHandlerWithInput(input, processor: step3))
}
}
func comboHandler(result: StepResult<Double>) {
switch result {
case .Success(let output):
print("output: \(output)")
case .Failure(let failHandler):
failHandler(comboHandler) // call again until success
}
}
let combinedSteps = step1 => step2 => step3
combinedSteps(5) { result in
comboHandler(result)
}
The answer to the specific question I asked here, in case it is useful to anyone else turned out to be to take a similar approach to Memoization. Strictly speaking, it is caching, rather than memoization, but the concept of wrapping the step function in another function that takes a cache key parameter is the same.
Before I added the feature to accept a input address all works fine.
After splitting of the IP-Address into 3 segments and surrender it to the getHostName function, the program skips "all/inclusive of the function" after the call of the function net.LookupAddr(ip).
package main
import (
"fmt"
"net"
"strconv"
"strings"
)
func getHostName(h chan string, ipAdresse string, n int) {
ip := ipAdresse + strconv.Itoa(n)
addr, ok := net.LookupAddr(ip)
fmt.Println(ok)
if ok == nil {
h <- ip + " - " + addr[0]
} else {
fmt.Println(ok)
}
}
func printer(n chan string) {
msg := <-n
fmt.Println(msg)
}
func main() {
fmt.Println("Please enter your local IP-Adresse e.g 192.168.1.1")
var ipAdresse_user string
fmt.Scanln(&ipAdresse_user)
ipsegment := strings.SplitAfter(ipAdresse_user, ".")
ipadresse_3 := ipsegment[0] + ipsegment[1] + ipsegment[2]
host := make(chan string)
for i := 0; i < 55; i++ {
go getHostName(host, ipadresse_3, i)
go printer(host)
}
fmt.Println("Finish - Network Scan")
}
My mistake i have to block the main function with e.g Scanln. Without it the program terminates before the goroutines can be executed.