It seems that URL does not support matrix parameters
// From net/url
type URL struct {
Scheme string
Opaque string // encoded opaque data
User *Userinfo // username and password information
Host string // host or host:port
Path string
RawQuery string // encoded query values, without '?'
Fragment string // fragment for references, without '#'
}
Why ?
How can I extract matrix parameters from an URL ? and when should I use them instead of using requests parameters embedded in the request.URL.RawQuery part of the URL ?
The parameters end up getting put in url.Path. Here's a function which can put them in the Query for you:
func ParseWithMatrix(u string) (*url.URL, error) {
parsed, err := url.Parse(u)
if err != nil {
return nil, err
}
if strings.Contains(parsed.Path, ";") {
q := parsed.Path[strings.Index(parsed.Path, ";")+1:]
m, err := url.ParseQuery(q)
if err != nil {
return nil, err
}
for k, vs := range parsed.Query() {
for _, v := range vs {
m.Add(k, v)
}
}
parsed.Path = parsed.Path[:strings.Index(parsed.Path, ";")]
parsed.RawQuery = m.Encode()
}
return parsed, nil
}
Related
I'm trying to add a place within app scope via google place API. For this I'm using golang. But I keep on getting no result, there is no error message.
Here is my code
```
type latlng struct {
lat, lng float64
}
type newPlace struct {
location latlng
accuracy int
name string
phone_number string
address string
types string
}
func main() {
requestUrl := "https://maps.googleapis.com/maps/api/place/add/json?key=<MYAPIKEY>"
obj := newPlace{
location: latlng{
lat: 52.1502824,
lng: 38.2643063,
},
name: "some field",
types: "storage",
}
bodyBytes, err := json.Marshal(&obj)
if err != nil {
panic(err)
}
body := bytes.NewReader(bodyBytes)
rsp, err := http.NewRequest("POST", requestUrl, body)
if err != nil {
log.Fatal(err)
}
defer rsp.Body.Close()
body_byte, err := ioutil.ReadAll(rsp.Body)
if err != nil {
panic(err)
}
fmt.Println(string(body_byte))
}
```
Here is the documentation that I followed. https://developers.google.com/places/web-service/add-place
I'm a bit new to golang, any help would be much appreciated.
FYI I wrote this article on this touchy topic (JSON data encoded into a POST body request in Go).
You're missing 4 things here:
the http.Client creation. Then you need to execute the request you're preparing with http.NewRequest by using client.Do.
add json fields to your struct and export variables contained in struct by capitalizing variables's first letters
set Content-Type to application/json
Google is expecting an array instead of a string in types, so I replace with an array containing 1 string (but you should adapt this depending on how many types you want to pass to Google)
Here is a working script:
type latlng struct {
Lat float64 `json:"lat"`
Lng float64 `json:"lng"`
}
type newPlace struct {
Location latlng `json:"location"`
Accuracy int `json:"accuracy"`
Name string `json:"name"`
PhoneNumber string `json:"phone_number"`
Address string `json:"address"`
Types [1]string `json:"types"`
}
func main() {
requestUrl := "https://maps.googleapis.com/maps/api/place/add/json?key=<your key>"
types := [1]string{"storage"}
obj := newPlace{
Location: latlng{
Lat: 52.1502824,
Lng: 38.2643063,
},
Name: "some field",
Types: types,
}
bodyBytes, err := json.Marshal(&obj)
if err != nil {
fmt.Println(err)
}
body := bytes.NewReader(bodyBytes)
client := &http.Client{}
req, err := http.NewRequest("POST", requestUrl, body)
req.Header.Add("Content-Type", "application/json")
if err != nil {
fmt.Println(err)
}
rsp, err := client.Do(req)
defer rsp.Body.Close()
body_byte, err := ioutil.ReadAll(rsp.Body)
if err != nil {
fmt.Println(err)
}
fmt.Println(string(body_byte))
}
Hope it's working now !
You're trying to marshall objects to JSON which have no exported fields, so the resulting JSON document is empty. Per the JSON documentation, it will only marshall exported fields (those whose names begin with a capital letter). Try:
type latlng struct {
Lat float64 `json:"lat"`
Lng float64 `json:"lng"`
}
type newPlace struct {
Location latlng `json:"location"`
Accuracy int `json:"accuracy"`
Name string `json:"name"`
PhoneNumber string `json:"phone_number"`
Address string `json:"address"`
Types string `json:"types"`
}
I need to send http request to https://some-domain.com/getsomething/?id=myID
I have url and need to add to it a query parameter. Here is my Go code
baseUrl := "https://some-domain.com"
relativeUrl := "/getsomething/"
url, _ := url.Parse(baseUrl)
url.Path = path.Join(url.Path, relativeUrl)
// add parameter to query string
queryString := url.Query()
queryString.Set("id", "1")
// add query to url
url.RawQuery = queryString.Encode()
// print it
fmt.Println(url.String())
In output I see this url: https://some-domain.com/getsomething?id=1
And this one is required: https://some-domain.com/getsomething/?id=1
You can see that there is no / character before ?.
Do you know how to fix it without manual string manipulations?
https://play.golang.org/p/HsiTzHcvlQ
You can use ResolveReference.
package main
import (
"fmt"
"log"
"net/url"
)
func main() {
relativeUrl := "/getsomething/"
u, err := url.Parse(relativeUrl)
if err != nil {
log.Fatal(err)
}
queryString := u.Query()
queryString.Set("id", "1")
u.RawQuery = queryString.Encode()
baseUrl := "https://some-domain.com"
base, err := url.Parse(baseUrl)
if err != nil {
log.Fatal(err)
}
fmt.Println(base.ResolveReference(u))
}
https://play.golang.org/p/BIU29R_XBM
Is there a way in Go to combine URL paths similarly as we can do with filepaths using path.Join()?
For example see e.g. Combine absolute path and relative path to get a new absolute path.
When I use path.Join("http://foo", "bar"), I get http:/foo/bar.
See in Golang Playground.
The function path.Join expects a path, not a URL. Parse the URL to get a path and join with that path:
u, err := url.Parse("http://foo")
if err != nil { log.Fatal(err) }
u.Path = path.Join(u.Path, "bar.html")
s := u.String()
fmt.Println(s) // prints http://foo/bar.html
Use the url.JoinPath function in Go 1.19 or later:
s, err := url.JoinPath("http://foo", "bar.html")
if err != nil { log.Fatal(err) }
fmt.Println(s) // prints http://foo/bar.html
Use ResolveReference if you are resolving a URI reference from a base URL. This operation is different from a simple path join: an absolute path in the reference replaces the entire base path; the base path is trimmed back to the last slash before the join operation.
base, err := url.Parse("http://foo/quux.html")
if err != nil {
log.Fatal(err)
}
ref, err := url.Parse("bar.html")
if err != nil {
log.Fatal(err)
}
u := base.ResolveReference(ref)
fmt.Println(u.String()) // prints http://foo/bar.html
Notice how quux.html in the base URL does not appear in the resolved URL.
ResolveReference() in net/url package
The accepted answer will not work for relative url paths containing file endings like .html or .img. The ResolveReference() function is the correct way to join url paths in go.
package main
import (
"fmt"
"log"
"net/url"
)
func main() {
u, err := url.Parse("../../..//search?q=dotnet")
if err != nil {
log.Fatal(err)
}
base, err := url.Parse("http://example.com/directory/")
if err != nil {
log.Fatal(err)
}
fmt.Println(base.ResolveReference(u))
}
A simple approach to this would be to trim the /'s you don't want and join. Here is an example func
func JoinURL(base string, paths ...string) string {
p := path.Join(paths...)
return fmt.Sprintf("%s/%s", strings.TrimRight(base, "/"), strings.TrimLeft(p, "/"))
}
Usage would be
b := "http://my.domain.com/api/"
u := JoinURL(b, "/foo", "bar/", "baz")
fmt.Println(u)
This removes the need for checking/returning errors
In 1.19 there will be a new function in the standard library that solves this very neatly.
u, err := url.JoinPath("http://host/foo", "bar/")
https://go.dev/play/p/g422ockBq0q?v=gotip
To join a URL with another URL or a path, there is URL.Parse():
func (u *URL) Parse(ref string) (*URL, error)
Parse parses a URL in the context of the receiver. The provided URL
may be relative or absolute. Parse returns nil, err on parse failure,
otherwise its return value is the same as ResolveReference.
func TestURLParse(t *testing.T) {
baseURL, _ := url.Parse("http://foo/a/b/c")
url1, _ := baseURL.Parse("d/e")
require.Equal(t, "http://foo/a/b/d/e", url1.String())
url2, _ := baseURL.Parse("../d/e")
require.Equal(t, "http://foo/a/d/e", url2.String())
url3, _ := baseURL.Parse("/d/e")
require.Equal(t, "http://foo/d/e", url3.String())
}
I wrote this utility function that works for my purposes:
func Join(basePath string, paths ...string) (*url.URL, error) {
u, err := url.Parse(basePath)
if err != nil {
return nil, fmt.Errorf("invalid url")
}
p2 := append([]string{u.Path}, paths...)
result := path.Join(p2...)
u.Path = result
return u, nil
}
https://play.golang.org/p/-QNVvyzacMM
I wanted to know how the container id is generated so please provide the source code that provides the container id when the docker run is executed?
Here is a code snippet from docker daemon's function for creating Containers:
func (daemon *Daemon) newContainer(name string, config *runconfig.Config, imgID string) (*Container, error) {
var (
id string
err error
)
id, name, err = daemon.generateIDAndName(name)
if err != nil {
return nil, err
}
…
base := daemon.newBaseContainer(id)
…
base.ExecDriver = daemon.execDriver.Name()
return &base, err
}
So, the logic of creating ID and Name is in generateIDAndName function:
func (daemon *Daemon) generateIDAndName(name string) (string, string, error) {
var (
err error
id = stringid.GenerateNonCryptoID()
)
if name == "" {
if name, err = daemon.generateNewName(id); err != nil {
return "", "", err
}
return id, name, nil
}
if name, err = daemon.reserveName(id, name); err != nil {
return "", "", err
}
return id, name, nil
}
Here is stringid sources and the concrete method is generateID with false as input parameter:
func generateID(crypto bool) string {
b := make([]byte, 32)
var r io.Reader = random.Reader
if crypto {
r = rand.Reader
}
for {
if _, err := io.ReadFull(r, b); err != nil {
panic(err) // This shouldn't happen
}
id := hex.EncodeToString(b)
// if we try to parse the truncated for as an int and we don't have
// an error then the value is all numberic and causes issues when
// used as a hostname. ref #3869
if _, err := strconv.ParseInt(TruncateID(id), 10, 64); err == nil {
continue
}
return id
}
}
As you can see, the value is randomly generated with this random
// Reader is a global, shared instance of a pseudorandom bytes generator.
// It doesn't consume entropy.
var Reader io.Reader = &reader{rnd: Rand}
I have trouble with memory. I don't understand why Go uses more and more memory (never freeing it) when my program runs for a long time.
After the first allocation, program uses nearly 9 MB of memory. Then after 12 hours it starts to use more memory exponentially, until 800 MB.
//.....code.....
if bol {
// Assignment Struct.Var
Struct_VastScript.TxtNoticeTop = JsonStruct_S.Options.TxtNoticeTop
Struct_VastScript.TxtNoticeBottom = JsonStruct_S.Options.TxtNoticeBottom
Struct_VastScript.Loop = JsonStruct_S.Options.Loop
Struct_Image, err := getImage(Struct_VastScript.Video)
if err == nil {
if mobile == "true" {
Struct_VastScript.Image = Struct_Image.URL360
}
}
//open and parse a template file
fi = path.Join("templates/VastPlayer", "TempVastPlayer.txt")
tmpl, err := template.ParseFiles(fi)
if err != nil {
job_1.Complete(health.Panic)
return false, err
}
//substitute fields in the template 'tmpl', with values from 'XmlStruct_V' and write it out to 'buf'
var buf bytes.Buffer
if err := tmpl.Execute(&buf, Struct_VastScript); err != nil {
//if err := tmpl.Execute(w, XmlStruct_V); err != nil {
job_1.Complete(health.Panic)
return false, err
}
// Call Func randString() : return alphanum random
dir := randString(12)
fpath := "http://creative2.xxx.io/api/html/" + dir
// Create a new EndPoint to write the generated 'template' on 'w' http.ResponseWriter
routeHtml := "/api/html/" + dir
http.HandleFunc(routeHtml, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
//writes Template to 'w' http.ResponseWriter
fmt.Fprintf(w, buf.String())
fmt.Println("successfull Operation 2 !!")
fmt.Println("")
job_2.Complete(health.Success)
}))
//Call Func JsonReply(): return the finale Json response
str := JsonReply(fpath, JsonStruct_S.Options.Animated, JsonStruct_S.Options.Responsive, JsonStruct_S.Options.Clickurl, JsonStruct_S.Options.Width, JsonStruct_S.Options.Height, adid, campaignid, JsonStruct_S.Type, JsonStruct_S.Options.Aspectratio, mobile)
w.Header().Set("Content-Type", "application/json")
//writes FinaleJson to 'w' http.ResponseWriter(it contains the link of the second endpoint "/api/html/")
fmt.Fprint(w, str)
fmt.Println("successfull Operation !!")
fmt.Println("")
job_1.Complete(health.Success)
return true, nil
} else {
return false, nil
}
For each call,my service need to generate a new template with the params that I receive,as you see I create a new endpoint for each call, I don't know if it's a good idea, I think the problem comes from this part of code but Im not sure because I don't know how GO manage it.
Obviously, you should not create handler every time request appears. They never free the memory so you will end up having out of memory exception.
Instead, put the handler endpoint into array (slice) and use ONE handler that responds to the request by looking the URL in this slice and then removing the item from the slice with it is not needed any longer.
So basically, instead of
routeHtml := "/api/html/" + dir
http.HandleFunc(routeHtml, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
//writes Template to 'w' http.ResponseWriter
fmt.Fprintf(w, buf.String())
fmt.Println("successfull Operation 2 !!")
fmt.Println("")
job_2.Complete(health.Success)
}))
do
type JobInfo struct {
Path string
// some data here
}
// maybe global context
var jobs []JobInfo
// initialisation
jobs = make([]JobInfo, 0)
http.HandleFunc("/api/html/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
path := r.URL.Path
var job *JobInfo
for _, j := range jobs {
if j.Path == path {
job = &j
break
}
}
if job != nil {
// handle job request here
}
}))
// and then in the jobs' loop
handlers = append(handlers, JobInfo{"/api/html/" + dir, ...})
It will work because:
Patterns name fixed, rooted paths, like "/favicon.ico", or rooted subtrees, like "/images/" (note the trailing slash). Longer patterns take precedence over shorter ones, so that if there are handlers registered for both "/images/" and "/images/thumbnails/", the latter handler will be called for paths beginning "/images/thumbnails/" and the former will receive requests for any other paths in the "/images/" subtree.
Do not forget to clean the array jobs, of course.
Instead of using slice it's better to use map
type JobInfo struct {
Path string
// some data here
}
// global context
var jobs map[string]JobInfo
// initialisation
jobs = make(map[string]JobInfoStruct)
http.HandleFunc("/api/html/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
path := r.URL.Path
var job JobInfoStruct
var ok bool
job, ok = jobs[path]
if ok {
// handle job request here
//then after delete the job
delete(jobs, path)
}
}))
// and then in the jobs' loop
pathVideo := "/api/html/" + dir
jobs[pathVideo] = JobInfoStruct{pathVideo, ...}