I would like to have the user upload a file inside a form and upon submission, open the file and read each line in order to grab information from it to store in the database. So far I am only able to make an object and extract bytes from it.
def uploadedFile = request.getFile('name_of_file')
This gets me a commonsMultipartFile, but I am not sure if I need to stream the file into a buffer, or save the file into a temp folder and then try to open it that way.
uploadedFile.getbytes() will give me an array [71, 101, 110, 101, 114, 97, 108, 13, 10, 45, 45, 45, 45, 45, 45, 45, 13, 10, 83, 116....] which is not in a form I can do anything with.
I need to be able to open the file and read it line by line. Can I do that with the commonsMultipartFile - its options seem kind of limited.
The CommonsMultipartFile has a getInputStream() method which will return an InputStream on which you can call any Groovy enhancements, for example:
request.getFile('name_of_file').inputStream.eachLine { line ->
/* Parsing logic */
}
Save it to a dir first. Then parse and do what you need done.
Related
I’m trying to import a course that has multiple videos and is hosted on an ironwood server to a newly deployed juniper server. While importing it throws the following errors in the console.
Following is the console log output
TypeError: Unicode-objects must be encoded before hashing
[2020-12-11 09:29:02,847: INFO/Worker-1] VAL: Video created with id [54454916-47e9-4769-8b41-06062d0b7e8c] and status [external]
[2020-12-11 09:29:02,860: ERROR/Worker-1] [VAL] Transcript save failed to storage for video_id “54454916-47e9-4769-8b41-06062d0b7e8c” language code “en”
Traceback (most recent call last):
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/edxval/models.py”, line 489, in create
video_transcript.transcript.save(file_name, transcript_content)
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/django/db/models/fields/files.py”, line 87, in save
self.name = self.storage.save(name, content, max_length=self.field.max_length)
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/django/core/files/storage.py”, line 52, in save
return self._save(name, content)
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/storages/backends/s3boto3.py”, line 495, in _save
self._save_content(obj, content, parameters=parameters)
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/storages/backends/s3boto3.py”, line 510, in _save_content
obj.upload_fileobj(content, ExtraArgs=put_parameters)
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/boto3/s3/inject.py”, line 513, in object_upload_fileobj
ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/boto3/s3/inject.py”, line 431, in upload_fileobj
return future.result()
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/s3transfer/futures.py”, line 73, in result
return self._coordinator.result()
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/s3transfer/futures.py”, line 233, in result
raise self._exception
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/s3transfer/tasks.py”, line 126, in call
return self._execute_main(kwargs)
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/s3transfer/tasks.py”, line 150, in _execute_main
return_value = self._main(**kwargs)
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/s3transfer/upload.py”, line 692, in _main
client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/client.py”, line 317, in _api_call
return self._make_api_call(operation_name, kwargs)
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/client.py”, line 596, in _make_api_call
request_signer=self._request_signer, context=request_context)
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/hooks.py”, line 242, in emit_until_response
responses = self._emit(event_name, kwargs, stop_on_response=True)
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/hooks.py”, line 210, in _emit
response = handler(**kwargs)
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/handlers.py”, line 209, in conditionally_calculate_md5
calculate_md5(params, **kwargs)
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/handlers.py”, line 187, in calculate_md5
binary_md5 = _calculate_md5_from_file(body)
File “/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/handlers.py”, line 201, in _calculate_md5_from_file
md5.update(chunk)
The course is imported from ironwood deployed sever to juniper deployed server.
I went through all the logs and was able to track that an error occurs in edxval while uploading the file to the S3 bucket. So, I checked edxval release versions and version 1.4.3 is the one that fixed S3 bucket upload issue. I updated it and it fixed my issue.
If my array goes over a certain length:
const List hts = [200, 195, 190, 185, 180, 175, 170, 165, 160, 155, 150, 145];
VSCode formats it like this:
const List hts = [
200,
195,
190,
185,
180,
175,
170,
165,
160,
155,
150,
145,
140
];
I'd rather it didn't since it makes a short array take up half the page. How can I change this?
Also I get even weirder formatting like this which means if I comment out a line it only comments out half the line:
This:
Text('RESULTS', style: kMainText),
Text('Additional', textAlign: TextAlign.center, style: kSmallText),
Gets formatted as this:
Text('RESULTS', style: kMainText),
Text('Additional',
textAlign: TextAlign.center, style: kSmallText),
The second line is one line but I have to comment it out as two lines. It seems like there's some kind of 'length' setting somewhere that I could alter to avoid this. How can I edit all this behaviour?
The Dart VS Code extension uses dart_style for formatting, which ships in the Dart SDK.
There is a setting called "maximum line length" which can be configured in your VS Code settings in the Dart section which will allow lines to be longer before they wrap (though note that this applies to all lines, not just lists).
There's an issue requesting an alternative (more customisable) formatter in VS Code here:
https://github.com/Dart-Code/Dart-Code/issues/914
Please add a 👍 to the issue if it's something you'd like to see.
The dart formatter will preserve line breaks in an list if there's a comment line at the head of this list.
For example, to format a very long list in Dart:
[
// yeah
a,b,c,
d,e,f,
];
See the test case (link) from official dart style repository (https://github.com/dart-lang/dart_style)
I want to use niftynet pretrained segmentation model for segmenting custom data. I downloaded the pre trained weights and and modified model_dir path to downloaded one.
However when I run
python3 net_segment.py train -c /home/Container_data/config/promise12_demo_train_config.ini
I am getting the error below.
Caused by op 'save/Assign_17', defined at:
File "net_segment.py", line 8, in <module>
sys.exit(main())
File "/home/NiftyNet/niftynet/__init__.py", line 142, in main
app_driver.run(app_driver.app)
File "/home/NiftyNet/niftynet/engine/application_driver.py", line 197, in run
SESS_STARTED.send(application, iter_msg=None)
File "/usr/local/lib/python3.5/dist-packages/blinker/base.py", line 267, in send
for receiver in self.receivers_for(sender)]
File "/usr/local/lib/python3.5/dist-packages/blinker/base.py", line 267, in <listcomp>
for receiver in self.receivers_for(sender)]
File "/home/NiftyNet/niftynet/engine/handler_model.py", line 109, in restore_model
var_list=to_restore, save_relative_paths=True)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/saver.py", line 1102, in __init__
self.build()
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/saver.py", line 1114, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/saver.py", line 1151, in _build
build_save=build_save, build_restore=build_restore)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/saver.py", line 795, in _build_internal
restore_sequentially, reshape)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/saver.py", line 428, in _AddRestoreOps
assign_ops.append(saveable.restore(saveable_tensors, shapes))
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/saver.py", line 119, in restore
self.op.get_shape().is_fully_defined())
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/state_ops.py", line 221, in assign
validate_shape=validate_shape)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_state_ops.py", line 61, in assign
use_locking=use_locking, name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1770, in __init__
self._traceback = tf_stack.extract_stack()
InvalidArgumentError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a mismatch between the current graph and the graph from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
Assign requires shapes of both tensors to match. lhs shape= [3,3,61,256] rhs shape= [3,3,3,61,9]
[[node save/Assign_17 (defined at /home/NiftyNet/niftynet/engine/handler_model.py:109) = Assign[T=DT_FLOAT, _class=["loc:#DenseVNet/conv/conv_/w"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](DenseVNet/conv/conv_/w, save/RestoreV2/_35)
https://github.com/tensorflow/models/issues/5390
Above link says to add
--initialize_last_layer = False
--last_layers_contain_logits_only = False
Can some one help me how to get rid of this error.
It seems you are having problems with your last layer. When you use a pretrained model on a new task you probably need to change your last layer to fit your new requirements.
In order to do that you should modify your config file by restoring all vars but last layer:
vars_to_restore = ^((?!(last_layer_name)).)*$
and then set num_classes to suit your new segmentation problem.
You can check transfer learning docs here: https://niftynet.readthedocs.io/en/dev/transfer_learning.html
I've had some working api code for quite a long time but suddenly (about 30 minutes from the previous use of the api) it's stopped working
here's the traceback
row_cells = self.range('%s:%s' % (start_cell, end_cell))
File"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/gspread/models.py", line 72, in wrapper
return method(self, *args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/gspread/models.py", line 412, in range
params={'range': name, 'return-empty': 'true'}
File "/Lbrary/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/gspread/client.py", line 176, in get_cells_feed
r = self.session.get(url)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/gspread/httpsession.py", line 73, in get
return self.request('GET', url, params=params, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/gspread/httpsession.py", line 69, in request
response.status_code, response.content))
gspread.exceptions.RequestError: (401, '401: b\'<HTML>\\n<HEAD>\\n<TITLE>Unauthorized</TITLE>\\n</HEAD>\\n<BODY BGCOLOR="#FFFFFF" TEXT="#000000">\\n<H1>Unauthorized</H1>\\n<H2>Error 401</H2>\\n</BODY>\\n</HTML>\\n\'')
And I don't really understand this...
here's the code
import gspread
from oauth2client.service_account import ServiceAccountCredentials
import pprint
scope = [ 'https://spreadsheets.google.com/feeds' ]
creds = ServiceAccountCredentials.from_json_keyfile_name('client_secret.json', scope)
client = gspread.authorize(creds)
sheet = client.open('sheet_name').sheet1
I really don't know what to do, I've already created new api's email address, downloaded the json file (client_secret.json) but it still didn't get back to working and I honestly don't know why
I frequently encounter this error in Mako templates using Pylons 0.9.7:
AttributeError: 'NoneType' object has no attribute 'decode'
Usually I've entered a variable name that doesn't exist, tried to use a linbebreak within a code line, or some other minor error. Definitely my fault.
This results in a 'Internal Server Error' in the browser, same thing in the debug view, and a stack trace that starts in HTTPServer and ends with the AttributeError in mako/exceptions.py.
Is there anything I can do to make this easier to debug, like find out the line that exception is being generated on within the Mako template? Thanks!
I am not absolutely sure that it's the same issue, but as far as I remember this used to happen a lot when you do AJAX loads of page fragments. Then you don't get anything more useful than this message.
However if you try to load the address of the AJAX request itself in your browser (replacing post parameters with get parameters if needed), you will get a "normal" debug page.
In my case, it turns out there was a division by 0 error in my template. That was yielding an internal server error, and a very unhelpful stack trace in console output.
I know, it sounds like I shouldn't have had that logic in a template anyway, but in this case, I think it makes sense to do so. Here's the stack trace I get with division by 0:
Exception happened during processing of request from ('127.0.0.1', 50681)
Traceback (most recent call last):
File "/Library/Python/2.6/site-packages/Paste-1.7.4-py2.6.egg/paste/httpserver.py", line 1068, in process_request_in_thread
self.finish_request(request, client_address)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/SocketServer.py", line 320, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/SocketServer.py", line 615, in __init__
self.handle()
File "/Library/Python/2.6/site-packages/Paste-1.7.4-py2.6.egg/paste/httpserver.py", line 442, in handle
BaseHTTPRequestHandler.handle(self)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/BaseHTTPServer.py", line 329, in handle
self.handle_one_request()
File "/Library/Python/2.6/site-packages/Paste-1.7.4-py2.6.egg/paste/httpserver.py", line 437, in handle_one_request
self.wsgi_execute()
File "/Library/Python/2.6/site-packages/Paste-1.7.4-py2.6.egg/paste/httpserver.py", line 287, in wsgi_execute
self.wsgi_start_response)
File "/Library/Python/2.6/site-packages/Paste-1.7.4-py2.6.egg/paste/cascade.py", line 130, in __call__
return self.apps[-1](environ, start_response)
File "/Library/Python/2.6/site-packages/Paste-1.7.4-py2.6.egg/paste/registry.py", line 375, in __call__
app_iter = self.application(environ, start_response)
File "/Library/Python/2.6/site-packages/Pylons-0.9.7-py2.6.egg/pylons/middleware.py", line 201, in __call__
self.app, environ, catch_exc_info=True)
File "/Library/Python/2.6/site-packages/Pylons-0.9.7-py2.6.egg/pylons/util.py", line 94, in call_wsgi_application
app_iter = application(environ, start_response)
File "/Library/Python/2.6/site-packages/WebError-0.10.2-py2.6.egg/weberror/evalexception.py", line 235, in __call__
return self.respond(environ, start_response)
File "/Library/Python/2.6/site-packages/WebError-0.10.2-py2.6.egg/weberror/evalexception.py", line 483, in respond
return debug_info.content()
File "/Library/Python/2.6/site-packages/WebError-0.10.2-py2.6.egg/weberror/evalexception.py", line 545, in content
result = tmpl_formatter(self.exc_value)
File "/Library/Python/2.6/site-packages/Pylons-0.9.7-py2.6.egg/pylons/error.py", line 43, in mako_html_data
css=False)
File "/Library/Python/2.6/site-packages/Mako-0.3.2-py2.6.egg/mako/template.py", line 189, in render
return runtime._render(self, self.callable_, args, data)
File "/Library/Python/2.6/site-packages/Mako-0.3.2-py2.6.egg/mako/runtime.py", line 403, in _render
_render_context(template, callable_, context, *args, **_kwargs_for_callable(callable_, data))
File "/Library/Python/2.6/site-packages/Mako-0.3.2-py2.6.egg/mako/runtime.py", line 434, in _render_context
_exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
File "/Library/Python/2.6/site-packages/Mako-0.3.2-py2.6.egg/mako/runtime.py", line 457, in _exec_template
callable_(context, *args, **kwargs)
File "memory:0x1040470d0", line 54, in render_body
File "/Library/Python/2.6/site-packages/Mako-0.3.2-py2.6.egg/mako/exceptions.py", line 88, in __init__
self.records = self._init(traceback)
File "/Library/Python/2.6/site-packages/Mako-0.3.2-py2.6.egg/mako/exceptions.py", line 166, in _init
line = line.decode('ascii', 'replace')
AttributeError: 'NoneType' object has no attribute 'decode'
----------------------------------------