uploading a file via a form/POST request in my Flask app works fine when the app is executed directly but fails when Shinyproxy hosts the Flask app. I tracked the issue down to the point that Shinyproxy is not executing the form POST request properly:
app.py:
from flask import Flask, request, render_template
app = Flask(__name__, static_url_path="/static")
#app.route('/', methods=['GET', 'POST'])
def index():
if request.method == 'GET':
return render_template('index.html')
else:
return "POST request received"
app.run(host='0.0.0.0', port=3838)
index.html:
<html>
<body>
<form method=post enctype=multipart/form-data>
<input type="file" name="file"/>
<input type = "submit" value="Upload">
</form>
</body>
</html>
returns, after clicking on "Upload", what one would expect, the GET and the POST request:
10.81.71.42 - - [23/Dec/2019 14:37:28] "GET / HTTP/1.1" 200 -
10.81.71.42 - - [23/Dec/2019 14:37:30] "POST / HTTP/1.1" 200 -
Running the exact same app in Shinyproxy via the
Dockerfile
FROM python:3
RUN pip install flask werkzeug
RUN mkdir /templates
COPY ["index.html", "/templates"]
COPY app.py /
EXPOSE 3838
CMD ["python", "app.py"]
returns the same but without the POST line:
172.17.0.1 - - [23/Dec/2019 14:39:42] "GET / HTTP/1.1" 200 -
The shinyproxy.log says
2019-12-23 14:42:33.682 DEBUG 17832 --- [XNIO-2 I/O-1] io.undertow.server.handlers.proxy : Sent request ClientRequest{path='/', method=POST, protocol=HTTP/1.1} to target 10.81.71.42 for exchange HttpServerExchange{ POST /proxy_endp
oint/b933863e-9fad-4d00-a657-034ede313e34/ request {Accept=[text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9], Accept-Language=[de-DE,de;q=0.9,en-US;q=0.8,en;q=0.
7], Cache-Control=[max-age=0], Accept-Encoding=[gzip, deflate], Origin=[http://192.168.76.81:8080], User-Agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36], Connectio
n=[keep-alive], Content-Length=[188], Content-Type=[multipart/form-data; boundary=----WebKitFormBoundarymPv6WtCTpGZQqRbF], Cookie=[JSESSIONID=CZxjGBM7BW597wysbBABgNWZL2x7qvsfujWVERgR], Referer=[http://192.168.76.81:8080/app_direct/flask_t
est/], Upgrade-Insecure-Requests=[1], Host=[192.168.76.81:8080]} response {Expires=[0], Cache-Control=[no-cache, no-store, max-age=0, must-revalidate], X-XSS-Protection=[1; mode=block], X-Content-Type-Options=[nosniff], Pragma=[no-cache]}
}
2019-12-23 14:42:33.683 DEBUG 17832 --- [XNIO-2 I/O-1] io.undertow.request.io : Fixed length stream closed with with 188 bytes remaining
2019-12-23 14:42:33.683 DEBUG 17832 --- [XNIO-2 I/O-1] i.u.client.http.HttpClientExchange : request terminated for request to localhost/127.0.0.1:20000 /
2019-12-23 14:42:33.683 ERROR 17832 --- [XNIO-2 I/O-1] io.undertow.proxy : UT005028: Proxy request to /proxy_endpoint/b933863e-9fad-4d00-a657-034ede313e34/ failed
io.undertow.server.TruncatedResponseException: null
Does anybody have an idea how I can host this app in shinyproxy (or otherwise)? I want to enable concurrent users to upload files and not using the same docker container/not interfere with each other.
Thanks and kind regards,
shosaco
That is a bug in Shinyproxy 2.2.0 and 2.3.0. Reverting to ShinyProxy 2.1.0 solves the problem, see https://github.com/openanalytics/shinyproxy/issues/184
Related
Context:
Sonarqube: 9.2.4.50792 --> https://sonar.dev.mycompany.com/
Gitlab: 13.4.3-ee --> http://git.mycomany.com/
What I am trying to achieve:
I try to delegate SonarQube authentication to Gitlab Self-Hosted.
What I've done:
Following the official SonarQube documentation, I've created a "GitLab OAuth app":
Went to http://git.mycompany.com/admin/applications
New Application:
Name: SonarQube
Redirect URI: https://sonar.dev.mycompany.com/oauth2/callback/gitlab
Trusted: Checked
Confidential: Checked
Scopes: all checked, to be sure my issue was not an access issue. (I'll tune it later)
I have copied both Application ID and Secret to use later.
Went to https://sonar.dev.mycompany.com/admin/settings?alm=gitlab&category=almintegration
GitLab Authentication:
Enabled: Yes, of course
GitLab URL: http://git.mycompany.com/users/auth/gitlab/callback
Application ID: The one given by GitLab during step 2
Secret: The one given by GitLab during step 2
Allow users to sign-up: Enabled
Synchronize user groups: Enabled
Disconnected to my admin account
The new Log in with GitLab is well shown:
When I clicked on this new button, I was well redirected to my GitLab instance (URL: http://git.mycompany.com/users/sign_in).
I was using my GitLab credentials then clicked on "Sign in"
I've got a GitLab 404 error page:
The URL was:
http://git.mycompany.com/users/auth/gitlab/callback/oauth/authorize?response_type=code&client_id=<MY_APPLICATION_ID>&redirect_uri=http%3A%2F%2Fsonar.dev.mycompany.com%3A9000%2Foauth2%2Fcallback%2Fgitlab&scope=api&state=ai1rq82joi504ggv3nc1qa0h29
The redirect_uri seems to be constructed with my sonar.properties informations (html_encoded):
"http://" + sonar.web.host + ":" sonar.web.port + "oauth2/callback/gitlab"
Investigation:
While I'm reproducing this error, I'm following (tail -f) some log files:
root#sonar.dev.mycompany.com: tail -f /var/log/httpd/*_log /var/log/message /home/sonar/sonarqube/logs/*.log
As expected, there is no information regarding the error 404 on GitLab.
On my GitLab server :
root#git.mycompany.com: tail -f /var/log/httpd/*_log /var/log/message /var/log/gitlab/*/current
And a message appears:
==> /var/log/gitlab/gitlab-workhorse/current <==
{
"correlation_id":"ZUVztnRFFe9",
"duration_ms":62,
"host":"git.mycompany.com",
"level":"info",
"method":"GET",
"msg":"access",
"proto":"HTTP/1.1",
"referrer":"",
"remote_addr":"127.0.0.1:0",
"remote_ip":"127.0.0.1",
"status":404,
"system":"http",
"time":"2022-01-19T18:06:47+01:00",
"uri":"/users/auth/gitlab/callback/oauth/authorize?response_type=code\u0026client_id=<MY_APPLICATION_ID>\u0026redirect_uri=http%3A%2F%2Fsonar.dev.mycompany.com%3A9000%2Foauth2%2Fcallback%2Fgitlab\u0026scope=api\u0026state=16r2cvc196v4fj8k0rv7oprs6h",
"user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36 Edg/97.0.1072.62",
"written_bytes":28281
}
Is there anyone who can help me?
Your Redirect URL is http and your sonarqube https... I hope you get the problem
I was installed this library, I am working with sails.js
npm install sails-hook-swagger-generator --save
from their main website https://github.com/theoomoregbee/sails-hook-swagger-generator
I have created this file ./swagger/swagger.json
I want to see the swagger from the browser, how can I see the endpoints?
I have in routes something like that
'GET /api/v1/information': { action: 'actions/getinformation' },
I can not see nothing from the browser, what is the problem?
EDIT 1
from their answer https://github.com/theoomoregbee/sails-hook-swagger-generator/issues/27
you can simply change the directory you want to place the generated
swagger json may be assets, then clone
https://github.com/swagger-api/swagger-ui following the instructions
here
Just simply extract the dist folder (which is needed for just the
swagger ui) and update this line
https://github.com/swagger-api/swagger-ui/blob/1a95b9e9c972b95227a2976553fea988e58d7ff2/dist/index.html#L44
to be the path to the generated swagger json within your assets folder
i dont know what should i put in that line, i am putting the path of my file swagger.json
/disk/team/work/proyect/swagger
but when i try this url in the browser
http://127.0.0.1:8080/
i am getting this [2020-09-01T19:39:35.791Z] "GET /" Error (404): "Not found"
[2020-09-01T19:39:35.961Z] "GET /" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36"
[2020-09-01T19:39:35.963Z] "GET /" Error (404): "Not found"
Not sure what I set wrong but I am not getting the graphiql interface when running in uvicorn using uvicorn mysite.asgi:application:
[32mINFO[0m: Started server process [[36m14872[0m]
[32mINFO[0m: Waiting for application startup.
[32mINFO[0m: ASGI 'lifespan' protocol appears unsupported.
[32mINFO[0m: Application startup complete.
[32mINFO[0m: Uvicorn running on [1mhttp://127.0.0.1:8000[0m (Press CTRL+C to quit)
[32mINFO[0m: 127.0.0.1:52463 - "GET /graphql/ HTTP/1.1" 200
Not Found: /static/graphene_django/graphiql.js
[33mWARNING[0m: Not Found: /static/graphene_django/graphiql.js
[32mINFO[0m: 127.0.0.1:52463 - "GET /static/graphene_django/graphiql.js HTTP/1.1" 404
Not Found: /static/graphene_django/graphiql.js
[33mWARNING[0m: Not Found: /static/graphene_django/graphiql.js
[32mINFO[0m: 127.0.0.1:52463 - "GET /static/graphene_django/graphiql.js HTTP/1.1" 404
but it loads fine when I do python manage.py runserver
Here is what I have installed:
Python 3.8.2
Django==3.0.5
uvicorn==0.11.3
graphene==2.1.8
graphene-django==2.9.0
graphql-core==2.3.1
In settings.py I have:
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATICFILES_DIRS = [os.path.join(BASE_DIR, "static"),]
# Graphene
GRAPHENE = {
'SCHEMA': 'mysite.schema.schema'
}
Just set DEBUG = True in your settings.py file.
Django is not serving static files by itself outside of the development server, you have to do it by yourself in some way. You can either set up a reverse proxy like nginx and instruct it to serve all the static files from your staticfiles directory or, only for development, you can instruct django to serve them, by adding to your urls.py:
from django.conf import settings
from django.views.static import serve
# Put the line provided below into your `urlpatterns` list.
url(r'^(?P<path>.*)$', serve, {'document_root': settings.STATIC_ROOT})
Make sure to use the later method only for the development, as it may have serious performance impact.
For both of the methods, you have to collect your static files as well using python manage.py collectstatic, as django, outside of the development server, needs all the static files to be collected in one place, your STATIC_ROOT.
I am sending logs to elasticsearch via rsyslog. For the parsing of those logs, I am using liblognorm rule.
I want to create fields of nginx logs,
here is a log entry,
127.0.0.1 - kibanaadmin [13/Jun/2017:14:18:17 +0530] "GET /ui/favicons/favicon-32x32.png HTTP/1.1" 304 0 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0"
Here is the pattern file,
version=2
rule=:%clientip:ipv4% - %user:word% [%timestamp:char-to:]%] %auth:word% "%verb:alpha% %request:word%" %response:number% %bytes:number% "%referrer:word"%" "%agent:char-to:{"extradata":"("}"
The reason for parsefailure is I believe due to the date-time format.
Can somebody help in creating a rule for parsing nginx logs ?
For the aforementioned Nginx access log line you need a rule like this:
version=2
rule=:%clientip:ipv4%%-:whitespace%-%-:whitespace%%user:word%%-:whitespace%[%timestamp:char-to{"extradata":"]"}%]%-:whitespace%"%verb:word%%-:whitespace%%request:word%%-:whitespace%%protocol:char-to{"extradata":"\""}%"%-:whitespace%%response_code:number{"format":"number"}%%-:whitespace%%bytes:number{"format":"number"}%%-:whitespace%"%referrer:char-to{"extradata":"\""}%"%-:whitespace%"%agent:char-to{"extradata":"\""}%"
which produces the following output (pretty printed):
{
"agent": "Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:53.0) Gecko\/20100101 Firefox\/53.0",
"referrer": "-",
"bytes": 0,
"response_code": 304,
"protocol": "HTTP\/1.1",
"request": "\/ui\/favicons\/favicon-32x32.png",
"verb": "GET",
"timestamp": "13\/Jun\/2017:14:18:17 +0530",
"user": "kibanaadmin",
"clientip": "127.0.0.1"
}
To test your log file along with a Liblognorm rule file more easily you can create a nginx.rule file for your rules and run the command:
lognormalizer -H -p -r nginx.rule < nginx.log
You can find more information about lognormalizer command here.
I am using Pow and Nginx to serve my Rails 4 app in development. A simple file upload is returning 500 error and the request is not reaching the Rails controller. I assume this is the case because there is no mention of the request in the Rails log. Without any mention of the error, I am not sure what is going wrong here.
I started with an Ajax file upload but replaced it with a simple form which is also not working.
Tried this
and then went to this
Current avatar.html.erb
<%= form_for #user, html: { multipart: true }, method: "post", url: '/settings/avatar/update', class: "", id: "update_avatar" do |f| %>
<%= f.file_field :avatar, class: 'js-upload-photo-button js-change-avatar-btn', accept: 'image/png,image/gif,image/jpeg,image/jpg' %>
<%= f.submit "Upload" %>
<% end %>
Request headers in Chrome
Expanded Request Headers section
nginx.log
127.0.0.1 - - [01/Feb/2014:11:28:26 +0530] "POST /settings/avatar/update HTTP/1.1" 500 643 "https://allotrop.dev/settings/avatar" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
pow access.log
[Sat Feb 01 2014 11:28:26 GMT+0530 (IST)] INFO [127.0.0.1] GET allotrop.dev /500.html
There are other questions here about Rails returning 500 error with no mention in log rails 500 error no production log entry
, Rails 3.2.13, 500 error in development with no log
and How to properly diagnose a 500 error (Rails, Passenger, Nginx, Postgres)
But these are one-off errors and do not seem related to the problem I am facing. It would be great if anyone can point me in the right direction.
Update 1
Relevant line from routes.rb
post '/settings/avatar/update', to: 'settings#update_avatar'
Update 2
Found my nginx error log. BTW, if you are using Homebrew, it is at /usr/local/Cellar/nginx/1.4.0/logs/error.log
nginx/error.log
2014/02/01 13:05:54 [crit] 8787#0: *85813 open() "/usr/local/var/run/nginx/client_body_temp/0000000010" failed (13: Permission denied), client: 127.0.0.1, server: *.dev, request: "POST /settings/avatar/update HTTP/1.1", host: "allotrop.dev", referrer: "https://allotrop.dev/settings/avatar"
The problem is not with Rails but with Nginx which is pretty evident from the nginx error.log. This question helped me understand what I was dealing with - Rails 3 + carrierwave + nginx = permission denied.
Nginx uses the client_body_temp_path directive to specify the location where it will temporarily store the uploaded files from the user request. Homebrew had set it by default to /usr/local/var/run/nginx. This folder also contains fastcgi_temp, proxy_temp, scgi_temp and uwsgi_temp for me. Nginx worker processes run with user nobody and they were not able to access these folders. I chowned all these folders to the nobody user, but that did not help.
Finally, I did
client_body_temp_path /tmp/nginx/; inside the HTTP module of my nginx.conf to make it work.
Doing a ls -l shows
drwx------ 2 nobody wheel 68 Feb 1 14:44 nginx
I am not sure why this worked inside /tmp and not inside the original /var/run/nginx. I belive I will face similar issue when I use other temp folders or in production. Will update this thread if and when that happens.
I recommend symlinking the other relevant logs like the nginx access and error log, pow access and app log to the /log directory of your Rails app. It helps in looking up errors in one of these when you face a tricky bug.