I am doing Machine Learning Course from Coursera by University of Washington. In which I am using iPython's graphlab. During practise when I execute below command:
sales = graphlab.SFrame('home_data.gl/')
I am getting error.
IOError Traceback (most recent call last)
<ipython-input-2-e6a249ea422b> in <module>()
----> 1 sales = graphlab.SFrame('home_data.gl/')
C:\Users\chinesh\Anaconda2\envs\gl-env\lib\site-packages\graphlab\data_structures\sframe.pyc in __init__(self, data, format, _proxy)
951 pass
952 else:
--> 953 raise ValueError('Unknown input type: ' + format)
954
955 sframe_size = -1
C:\Users\chinesh\Anaconda2\envs\gl-env\lib\site-packages\graphlab\cython\context.pyc in __exit__(self, exc_type, exc_value, traceback)
47 if not self.show_cython_trace:
48 # To hide cython trace, we re-raise from here
---> 49 raise exc_type(exc_value)
50 else:
51 # To show the full trace, we do nothing and let exception propagate
IOError: C:\Users\chinesh\home_data.gl not found.
where i can find home_data.gl in my computer or the problem is something else ..
You need to have your ipynb file and the data file in the same directory for the above to work. Alternatively, specify the full or relative path of the data file in the function
sales = graphlab.SFrame('C:\FULL-PATH\home_data.gl/')
Here is a link to the course reading for how to arrange your directories for the course. https://www.coursera.org/learn/ml-foundations/supplement/IT04V/reading-where-should-my-files-go
Make sure to download the zip folder to the same folder where you will work on this data. For example: I downloaded the zip file to Downloads then I opened my notebook from the download file
Just follow the instructions and download the training data into the ipython working directory.
Go to terminal and run:
unzip home_data.gl.zip
You will see following files in directory home_data.gl:
Now in ipython, run:
sales = graphlab.SFrame('home_data.gl/')
sales
which will display the data in tabular format:
Related
I am trying to run a cell in Jupyter notebook and getting permission error number 13. The code is
df_manual_testing = pd.concat([df_fake_manual_testing,df_true_manual_testing], axis = 0)
df_manual_testing.to_csv("manual_testing.csv")
The error which I am getting is:
---------------------------------------------------------------------------
PermissionError Traceback (most recent call last)
<ipython-input-16-f14a4d175882> in <module>
1 df_manual_testing = pd.concat([df_fake_manual_testing,df_true_manual_testing], axis = 0)
----> 2 df_manual_testing.to_csv("manual_testing.csv")
~\anaconda\lib\site-packages\pandas\core\generic.py in to_csv(self, path_or_buf, sep, na_rep, float_format, columns, header, index, index_label, mode, encoding, compression, quoting, quotechar, line_terminator, chunksize, date_format, doublequote, escapechar, decimal)
3202 decimal=decimal,
3203 )
-> 3204 formatter.save()
3205
3206 if path_or_buf is None:
~\anaconda\lib\site-packages\pandas\io\formats\csvs.py in save(self)
182 close = False
183 else:
--> 184 f, handles = get_handle(
185 self.path_or_buf,
186 self.mode,
~\anaconda\lib\site-packages\pandas\io\common.py in get_handle(path_or_buf, mode, encoding, compression, memory_map, is_text)
426 if encoding:
427 # Encoding
--> 428 f = open(path_or_buf, mode, encoding=encoding, newline="")
429 elif is_text:
430 # No explicit encoding
PermissionError: [Errno 13] Permission denied: 'manual_testing.csv'
I cannot understand how to change the permission of my file. I think so I will need to change it from root to user, but I am not sure exactly how to do this.
If you are not interested in writing the csv file into the current working directory, you could simply specify the full path of a folder in which you are sure you have the permissions to write.
For example:
df_manual_testing.to_csv("C:/Users/../manual_testing.csv")
However, if you want to write in a particular folder, you can check from the terminal if you have the permissions to write here, using the command ls -lh. Eventually, you could change the permissions using the account of the owner of the folder, with the command chmod 777 myfolder.
If you need more information about file permissions, you could look at this reference.
I am using a Flask server to handle requests for some image-processing tasks.
The processing relies extensively on OpenCV and I would now like to trivially-parallelize some of the slower steps.
I have a preference for multiprocessing rather than multithreading (please assume the former in your answers).
But multiprocessing with opencv is apparently broken (I am on Python 2.7 + macOS): https://github.com/opencv/opencv/issues/5150
One solution (see https://github.com/opencv/opencv/issues/5150#issuecomment-400727184) is to use the excellent Loky (https://github.com/tomMoral/loky)
[Question: What other working solutions exist apart from concurrent.futures, loky, joblib..?]
But Loky leads me to the following stacktrace:
a,b = f.result()
File "/anaconda2/lib/python2.7/site-packages/loky/_base.py", line 433, in result
return self.__get_result()
File "/anaconda2/lib/python2.7/site-packages/loky/_base.py", line 381, in __get_result
raise self._exception
BrokenProcessPool: A task has failed to un-serialize. Please ensure that the arguments of the function are all picklable.
This was caused directly by
'''
Traceback (most recent call last):
File "/anaconda2/lib/python2.7/site-packages/loky/process_executor.py", line 391, in _process_worker
call_item = call_queue.get(block=True, timeout=timeout)
File "/anaconda2/lib/python2.7/multiprocessing/queues.py", line 135, in get
res = self._recv()
File "myfile.py", line 44, in <module>
app.config['EXECUTOR_MAX_WORKERS'] = 5
File "/anaconda2/lib/python2.7/site-packages/werkzeug/local.py", line 348, in __getattr__
return getattr(self._get_current_object(), name)
File "/anaconda2/lib/python2.7/site-packages/werkzeug/local.py", line 307, in _get_current_object
return self.__local()
File "/anaconda2/lib/python2.7/site-packages/flask/globals.py", line 52, in _find_app
raise RuntimeError(_app_ctx_err_msg)
RuntimeError: Working outside of application context.
This typically means that you attempted to use functionality that needed
to interface with the current application object in some way. To solve
this, set up an application context with app.app_context(). See the
documentation for more information.
'''
The functions to be parallelized are not being called from app/main.py, but rather from an abitrarily-deep submodule.
I have tried the similarly-useful-looking https://flask-executor.readthedocs.io/en/latest, also so far in vain.
So the question is:
How can I safely pass the application context through to the workers or otherwise get multiprocessing working (without recourse to multithreading)?
I can build out this question if you need more information. Many thanks as ever.
Related resources:
Copy flask request/app context to another process
Flask Multiprocessing
Update:
Non-opencv calls work fine with flask-executor (no Loky) :)
The problem comes when trying to call an opencv function like knnMatch.
If Loky fixes the opencv issue, I wonder if it can be made to work with flask-executor (not for me, so far).
I want to install IPCV for Scilab but I get the error message:
-->atomsInstall('IPCV') atomsInstallList: The package IPCV is not available. !--error 10000
at line 51 of function atomsError called by : at line 76
of function atomsInstallList called by : at line 233 of function
atomsInstall called by : atomsInstall('IPCV')
what is causing the error? what should be done? OS is UBUNTU 14.04.
If I summarize and complete the comments above, you should
download Scilab 6.0.2 at https://www.scilab.org/download/6.0.2 (below I suppose that you took the 32 bits version)
extract the archive scilab-6.0.2.bin.linux-i686.tar.gz in your home directory (let us consider it as /home/user543265)
create a text file (e.g. with gedit) named scilab.desktop with the following content (replace user543265 by your real user id) and save it in /home/user543265/Desktop
[Desktop Entry]
Name=scilab-6.0.2
Icon=scilab
Exec=/home/user543265/scilab-6.0.2/bin/scilab
Terminal=false
Type=Application
Then you should be able to launch Scilab by double-clicking its icon on the Desktop. On the scilab command line you should be able to install IPCV by typing
atomsInstall IPCV
I've inherited an app from another developer and have added an error notification service, and have now received 1,408 occurrences of an error that I am having trouble getting to the bottom of...
The error is simply:
TypeError: can't dump MatchData
with only non-project frames in the stack trace....
I can see where this error is ultimately occurring in the activesupport gem message_verifier.rb file, but I assume that there's an error in the application or configuration that is causing it originally, and am not sure the best approach how to trace this back to its point of origin on our end. Strikes me as the kind of thing that someone may look at and immediately see recognize it. And advice appreciated....
This is the stack trace:
File "xxx/shared/bundle/ruby/1.9.1/gems/activesupport-3.2.22.2/lib/active_support/message_verifier.rb" line 53 in dump
File "xxx/shared/bundle/ruby/1.9.1/gems/activesupport-3.2.22.2/lib/active_support/message_verifier.rb" line 53 in generate
File "xxx/shared/bundle/ruby/1.9.1/gems/actionpack-3.2.22.2/lib/action_dispatch/middleware/cookies.rb" line 300 in []=
File "xxx/shared/bundle/ruby/1.9.1/gems/actionpack-3.2.22.2/lib/action_dispatch/middleware/session/cookie_store.rb" line 64 in set_cookie
File "xxx/shared/bundle/ruby/1.9.1/gems/rack-1.4.7/lib/rack/session/abstract/id.rb" line 335 in commit_session
File "xxx/shared/bundle/ruby/1.9.1/gems/rack-1.4.7/lib/rack/session/abstract/id.rb" line 211 in context
File "xxx/shared/bundle/ruby/1.9.1/gems/rack-1.4.7/lib/rack/session/abstract/id.rb" line 205 in call
File "xxx/shared/bundle/ruby/1.9.1/gems/actionpack-3.2.22.2/lib/action_dispatch/middleware/cookies.rb" line 341 in call
File "xxx/shared/bundle/ruby/1.9.1/gems/activerecord-3.2.22.2/lib/active_record/query_cache.rb" line 64 in call
File "xxx/shared/bundle/ruby/1.9.1/gems/activerecord-3.2.22.2/lib/active_record/connection_adapters/abstract/connection_pool.rb" line 479 in call
File "xxx/shared/bundle/ruby/1.9.1/gems/actionpack-3.2.22.2/lib/action_dispatch/middleware/callbacks.rb" line 28 in block in call
File "xxx/shared/bundle/ruby/1.9.1/gems/activesupport-3.2.22.2/lib/active_support/callbacks.rb" line 405 in _run__656928281__call__438689938__callbacks
File "xxx/shared/bundle/ruby/1.9.1/gems/activesupport-3.2.22.2/lib/active_support/callbacks.rb" line 405 in __run_callback
File "xxx/shared/bundle/ruby/1.9.1/gems/activesupport-3.2.22.2/lib/active_support/callbacks.rb" line 385 in _run_call_callbacks
File "xxx/shared/bundle/ruby/1.9.1/gems/activesupport-3.2.22.2/lib/active_support/callbacks.rb" line 81 in run_callbacks
File "xxx/shared/bundle/ruby/1.9.1/gems/actionpack-3.2.22.2/lib/action_dispatch/middleware/callbacks.rb" line 27 in call
File "xxx/shared/bundle/ruby/1.9.1/gems/actionpack-3.2.22.2/lib/action_dispatch/middleware/remote_ip.rb" line 31 in
While running the PyBBIO examples phant_test.py and analog_test.py I received the following error (I believe 'could' is a typo meant to be 'could not'):
Traceback (most recent call last):
File "analog_test.py", line 47, in <module>
run(setup, loop)
File "/usr/lib/python2.7/site-packages/PyBBIO-0.9-py2.7-linux-armv7l.egg/bbio/bbio.py", line 63, in run
loop()
File "analog_test.py", line 37, in loop
val1 = analogRead(pot1)
File "/usr/lib/python2.7/site-packages/PyBBIO-0.9-py2.7-linux-armv7l.egg/bbio/platform/beaglebone/bone_3_8/adc.py", line 46, in analogRead
raise Exception('*Could load overlay for adc_pin: %s' % adc_pin)
Exception: *Could load overlay for adc_pin: ['/sys/devices/ocp.2/PyBBIO-AIN0.*/AIN0', 'PyBBIO-AIN0', 'P9.39']
I have tried restarting the BeagleBone (rev A6 running Angstrom with a 3.8 kernel, with no capes connected) to clear the /sys/devices/bone_capemgr.7/slots file, but that did not work. It seems PyBBIO is accessing the slots file and adding overlays because the slots file looks like this after the example program runs:
0: 54:PF---
1: 55:PF---
2: 56:PF---
3: 57:PF---
4: ff:P-O-L Override Board Name,00A0,Override Manuf,PyBBIO-ADC
5: ff:P-O-L Override Board Name,00A0,Override Manuf,PyBBIO-AIN0
Since there were some changes being made to the slots file I checked what files the analog_read(adc_pin) function in the adc.py file of PyBBIO was retrieving. With some print statements I figured out the root problem was that the /sys/devices/ocp.2/PyBBIO-AIN0.*/AIN0 file, which apparently stores the analog read values, does not exist. The glob.glob function returns a null array, and ls /sys/devices/ocp.2/PyBBIO-AIN0.10/ shows modalias power subsystem uevent as the only contents. Is there something wrong in the overlay file? Or could there be another program or problem that is preventing the BeagleBone from writing the AIN0 file that PyBBIO is trying to read? The python code seems to be logically correct, but the overlay is working incorrectly or being blocked in some way.