I want to download a bunch of url list by:
urllib.request.urlretrieve(url, filename)
but sadly, few links are broken. and when urlretrieve meets that broken link
Traceback (most recent call last):
File "D:/Users/hyungsoo/PycharmProjects/untitled/check.py", line 71, in <module>
....blah_blah....
urllib.error.HTTPError: HTTP Error 404: Not Found
give me this error sign and program exited.
how to pass the broken url?
and moreover, is it possible to program tells me what link is broken?
You could try a "try...except..." statement, which you may find helpful. The "try" portion will attempt the given code, but the
"except" portion will be ready for the error message. It is really a wonderful feature. See teh following example:
while True:
... try:
... x = int(raw_input("Please enter a number: "))
... break
... except ValueError:
... print "Oops! That was no valid number. Try again..."
Note: Code taken from python APIs.
Related
Code used from github: https://github.com/opencontainers/runc/blob/master/libcontainer/rootfs_linux.go#L59
This is the if block to debug for our interest....
if err := mountToRootfs(m, config.Rootfs, config.MountLabel, hasCgroupns); err != nil {
return newSystemErrorWithCausef(err, "mounting %q to rootfs at %q", m.Source, m.Destination)
}
Here in the if condition, there is an error generated thru newSystemErrorWithCausef function and which internally uses generic_error.go, fmt.Sprintf to format the error string.
In our integration, only once it hit this if block. And in the process of getting more details, I wanted to troubleshoot and understand why this happened.
As of now tried using fmt.Printf(........) in line 55 and also just before if block of line 59.
And also tried putting above statement in very first line of prepareRootfs function. And then tried newSystemErrorWithCause(err, "some other message") in the first line of prepareRootfs function. Overall I dont see any information generated with my messages.
Please provide any information to understand more of this issue. Please note I am not looking for why this error might have been caused. I am looking for how to get more information from that function of prepareRootfs.
I've just started learning my way around Biopython and I'm trying to use ExPASy to retrieve SwissProt records, just like described in page 180 of the Biopython tutorial (http://biopython.org/DIST/docs/tutorial/Tutorial.pdf), but also in a relevant ROSALIND exercise (http://rosalind.info/problems/dbpr/ - click to expand the "Programming shortcut" section).
The code I'm using is basically the same as in the ROSALIND exercise:
from Bio import ExPASy
from Bio import SwissProt
handle = ExPASy.get_sprot_raw('Q5SLP9')
record = SwissProt.read(handle)
However, the SwissProt.read function gives the following error messages (I've trimmed some of the filepaths):
Traceback (most recent call last): File "code.py", line 4, in <module>
record = SwissProt.read(handle) File "lib\site-packages\Bio\SwissProt\__init__.py", line 151, in read
record = _read(handle) File "lib\site-packages\Bio\SwissProt\__init__.py", line 255, in _read
_read_ft(record, line) File "lib\site-packages\Bio\SwissProt\__init__.py", line 594, in _read_ft
assert not from_res and not to_res, line AssertionError: /note="Single-stranded DNA-binding protein"
I found this has been reported in GitHub (https://github.com/biopython/biopython/issues/2417), so I'm not the first one who gets this, but I don't really find any updated version of the package or any way to fix the issue. Maybe it's because I'm very new to using packages. Could someone help me please?
Please update your BioPython to version 1.77. The issue has been fixed with pull request 2484.
As I followed the steps to download CSV of problem responses for a problem, It says “The problem responses report is being created. To view the status of the report, see Pending Tasks below.”
But, I am not seeing any pending tasks nor the files are being generated. Can’t see the generated file even after refreshing.
But, I can generate other CSV files for example, Problem Grade Report.
The issue is only for the problem response report.
PS: When I checked Admin, I could see that the request was failed with this error:
{"exception": "AssertionError", "traceback": "Traceback (most recent call last):\n File \"/openedx/venv/local/lib/python2.7/site-packages/celery/app/trace.py\", line 240, in trace_task\n R = retval = fun(*args, **kwargs)\n File \"/openedx/venv/local/lib/python2.7/site-packages/celery/app/trace.py\", line 438, in __protected_call__\n return self.run(*args, **kwargs)\n File \"/openedx/edx-platform/lms/djangoapps/instructor_task/tasks.py\", line 171, in calculate_problem_responses_csv\n return run_main_task(entry_id, task_fn, action_name)\n File \"/openedx/edx-platform/lms/djangoapps/instructor_task/tasks_helper/runner.py\", line 111, in run_main_task\n task_progress = task_fcn(entry_id, course_id, task_input, action_name)\n File \"/openedx/edx-platform/lms/djangoapps/instructor_task/tasks_helper/grades.py\", line 737, in generate\n usage_key_str=problem_location\n File \"/openedx/edx-platform/lms/djangoapps/instructor_task/tasks_helper/grades.py\", line 674, ...", "message": ""}
Please help.
After having much tries I was able to figure out my issue was caused because one operation which is not quite preferable by Open Edx. It is changing correct answer of a problem after few students submitting their responses. I was able to get their grades calculated based on new answers by using the Re-score option but unable to generate this report.
I will update here if I get a solution.
python3.8
My code:
from googleads import adwords
def execute_request():
adwords_client = adwords.AdWordsClient.LoadFromStorage(path="google_general/googleads.yaml")
campaign_service = adwords_client.GetService('CampaignService', version='v201809')
pass
context["dict_list"] = execute_request()
Traceback:
Traceback (most recent call last):
File "/home/michael/pycharm-community-2019.3.2/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_xml.py", line 282, in frame_vars_to_xml
xml += var_to_xml(v, str(k), evaluate_full_value=eval_full_val)
File "/home/michael/pycharm-community-2019.3.2/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_xml.py", line 369, in var_to_xml
elif hasattr(v, "__len__") and not is_string(v):
File "/home/michael/PycharmProjects/ads3/venv/lib/python3.8/site-packages/googleads/common.py", line 694, in __getattr__
raise googleads.errors.GoogleAdsValueError('Service %s not found' % attr)
googleads.errors.GoogleAdsValueError: Service __len__ not found
Unexpected error, recovered safely.
googleads.yaml about logging
logging:
version: 1
disable_existing_loggers: False
formatters:
default_fmt:
format: ext://googleads.util.LOGGER_FORMAT
handlers:
default_handler:
class: logging.StreamHandler
formatter: default_fmt
level: DEBUG
loggers:
# Configure root logger
"":
handlers: [default_handler]
level: DEBUG
I've just started studying the API.
Namely, I'm trying to execute my first request (https://developers.google.com/adwords/api/docs/guides/first-api-call#make_your_first_api_call)
Could you help me with this problem? At least how to localize it more precisely.
This seems to be a problem which results from the way the PyCharm debugger inspects live objects during debugging.
Specifically, it checks if a given object has the __len__ attribute/method in the code of var_to_xml, most likely to determine an appropriate representation of the object for the debugger interface (which seems to require constructing an XML representation).
googleads service objects such as your campaign_service, however, use some magic to be able to call the defined SOAP methods on them without requiring to hard-code all of them. The code looks like this:
def __getattr__(self, attr):
"""Support service.method() syntax."""
if self._WsdlHasMethod(attr):
if attr not in self._method_proxies:
self._method_proxies[attr] = self._CreateMethod(attr)
return self._method_proxies[attr]
else:
raise googleads.errors.GoogleAdsValueError('Service %s not found' % attr)
This means that the debugger's check for a potential __len__ attribute is intercepted, and because the CampaignService does not have a SOAP operation called __len__, an exception is raised.
You can validate this by running your snippet in the regular way (i.e. not debugging it) and checking if that works.
An actual fix would seem to either require that PyCharm's debugger changes the way it inspects objects (not calling hasattr(v, "__len__")) or that googleads modifies the way it implements __getattr__, for example by actually implementing a __len__ method that just raises AttributeError.
I am trying to follow the sample code of the book "Mining the social web", 1-3.
I know its old so I follow the new sample from the web page enter link description here
BUT, SOMETIMES, I will suffer a Error info when I implement the code:
[ trend.decode('utf-8') for trend in world_trends()[0]['trends'] ]
And the error info is like this:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build/bdist.macosx-10.6-universal/egg/twitter/api.py", line 167, in __call__
File "build/bdist.macosx-10.6-universal/egg/twitter/api.py", line 173, in _handle_response
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x8b in position 1: unexpected code byte
It doesnt always happen, but I think no programmer likes such a "random" case.
So could anyone help me on this issue? Whats the problem and how I can solve this?
Great thanks~
byte 0x8b in position 1 usually signals that the data stream is gzipped. For similar problems, see here and here.
To unzip the data stream:
buf = StringIO.StringIO(<response object>.content)
gzip_f = gzip.GzipFile(fileobj=buf)
content = gzip_f.read()
By default decode() will throw an error if it encounters a byte that it doesn't know how to decode.
You can use trend.decode('utf-8', 'replace') or trend.decode('utf-8', 'ignore') to not throw an error and silently ignore it.
Documentation on decode() here.