I'm trying to simply open a file in pyfilesystem's s3 filesystem as such:
>>> s3fs.listdir('dfparks/test')
['test.txt']
>>> s3fs.open('dfparks/test/test.txt')
I get the error:
fs.errors.ResourceNotFound: resource 'dfparks/test/test.txt' not found
I've tried every variant of the path / s3 url I can think of but I always get the not found error.
fs.s3fs by default works in strict mode, checking for existence of directory markers (keys ending with "/"). You can turn off strict mode by passing strict=False to S3FS constructor or append "strict=0" to url passed to fs.open_fs
But, unfortunately, S3FS still does these checks in few places even in strict=False mode. There is my PR waiting for review fixing that: https://github.com/PyFilesystem/s3fs/pull/60. In the meantime you can install fixed version using fs-s3fs-ng package:
pip install fs-s3fs-ng
Additionally it enables non-strict mode by default and turns on S3 streaming (other PR waiting for acceptance)
Please let me know if this works for you.
Related
When using Apache Beam (GCP Dataflow) I see the following warning in worker logs:
No session file found: /var/opt/google/dataflow/pickled_main_session.
Functions defined in __main__ (interactive session) may fail.
My Dataflow job seems to be fine regardless, but I'm wondering what this warning is all about.
I have seen the following in some sample code (which I am NOT currently doing):
pipeline_options.view_as(SetupOptions).save_main_session = True
where pipeline_options is the main way of specifying options for the Beam/Dataflow pipeline, as in the following later in the code:
with beam.Pipeline(options=pipeline_options) as p:
# actual pipeline code here
I am curious if the two are related. Does the presence of the warning mean I should always be saving the main session? Are these two things related? Unrelated?
You should be able to safely ignore this warning. No need to set save_main_session if it's not required for your pipeline.
I'm trying to separate out some code from drake/automotive/automotive_demo.cc. As a first step, I'm trying to copy automotive_demo.cc and automotive_demo.py into differently named files (test.cc and test.py) and then running bazel run automotive:test -- --num_simple_cars=1. I modified automotive/BUILD.bazel and test.py to take into account the new dependencies.
The problem is that after I bazel run, the simulator window opens but no car gets rendered. Eventually it just crashes with the following errors:
[lcm-spy] ClassDiscoverer: java.lang.NoClassDefFoundError: apple/laf/AquaPopupMenuUI
[lcm-spy] jar: ../com_jidesoft_jide_oss/jide-oss-2.9.7.jar
[lcm-spy] class: com/jidesoft/plaf/aqua/AquaJidePopupMenuUI.class
...
[drake_visualizer] Qt WebEngine seems to be initialized from a plugin. Please set Qt::AA_ShareOpenGLContexts using QCoreApplication::setAttribute before constructing QGuiApplication.
...
[lcm-spy] LCM: Disabling IPV6 support
[lcm-spy] LCM: TTL set to zero, traffic will not leave localhost.
[lcm-spy] java.net.SocketException: Can't assign requested address
Here is an (unresolved) Github issue that points to the problem being that test is a "custom plug-in". But if automotive_demo can work, surely there's a way to reproduce that behavior for test? I also tried grepping for QGuiApplication and only found a series of binary files, so I didn't know how to follow the error message's suggestion.
when trying out your steps on Mac I unfortunately cannot reproduce your specific errors. I do not think that having test as a target name should cause problems (at least I did not experience issues).
Could you please make sure:
You're able to run bazel run automotive:demo -- --num_simple_car=1?
After having renamed automotive_demo.* to test.*, in your BAZEL.build, test.py files the following are mapped correctly: demo -> test and automotive_demo -> test_cc (or whatever unique name you choose)?
If I execute a binary in a clearcase view, and look at /proc/self/exe for that on Linux, I see something like the following:
$ cd /proc/19220
$ ls -l exe
lrwxrwxrwx 1 peeterj pdxdb2 0 2012-11-30 13:04 exe -> /home/peeterj/views/peeterj_clang-7.vws/.s/00024/8000028250b8f1d1llvm-config
The clang llvm-config program, not unreasonably, uses this output to try to figure out the absolute fully qualified path that it is located in (I assume in case argv[0] isn't fully qualified).
Is there a way to find the location within the view that this corresponds to. For example, in this case, the llvm-config exe is actually in:
/vbs/bldsupp/linuxamd64/clang/debug/bin
(I'm wondering if it's feasible to modify clang's GetExecutablePath() function to figure this out.)
No trivial solution here (for an old version of ClearCase though):
The technote "PK27447: WITHIN A CLEARCASE DYNAMIC VIEW, THE READLINK() CALL ON LINUX RETURNS THE WRONG PATH FOR THE EXECUTABLE'S /PROC/SELF/EXE VALUE" suggests:
Local fix
Use getcwd(), get_current_dir_name(), getwd() in applications slated for a VOB/View context
Create an interposer library to intercept the readlink() call, and modify to use any of the above calls to return the proper data
The cause:
/proc/self/exe returns the improper path while getcwd succeeds.
Unfortunately, for /proc/self/exe to return the proper value [from within a VOB/View context] would require a change within the Linux kernel to allow MVFS to "override" the present setting.
IBM LTC has been working on having the Linux community adopt this change so that we can then incorporate the new features within MVFS.
Related: Bug Sun 6189256.
I am extremely new to Informix and am having some trouble trying to get sqldemo installed.
Set up so far:
openSuse 12.1 (32 bit)
Informix Growth Edition 11.70 UC6
Informix SQL Developer 7.50 UC6
Informix RDS 7.50 UC6
Informix ID 7.50 UC6
After struggling a few days and a lot of reading of http://publib.boulder.ibm.com/infocenter/idshelp/v117/index.jsp, I managed to get Informix installed and On-line.
I also opted to install the demo database instance that comes with the installation.
I now and attempting to get started with Informix 4GL by Example.
I am trying to get the sqldemo database up. I don't know if it will replace the previous instance installed with Informix, but that is a different problem.
Right now as per the document, running the following should set up the DB:
sqldemo stores2t -log
I however get an error: "Invalid Locales set !!".
I have tried looking up this error and also in the documentation.
I have tried setting the CLIENT_LOCALE and DB_LOCALE in my .profile file.
For example:
export CLIENT_LOCALE=en_US.CP1252 and
export DB_LOCALE=en_US.819
This has not helped.
A push in the right direction, or perhaps some other documentation I could read that would explain things better would really be appreciated.
If any other information is required from me, please do not hesitate to ask.
Update 1
Thanks so much for the response.
A couple of things firstly that I have tried since your post.
Changed the the CLIENT_LOCALE and DB_LOCALE as you specified - Same error - So i removed it as you said it should not be set.
Fixed a problem in my PATH and made sure it has /usr/informix/bin - Same Error
INFORMIXDIR is /usr/informix
INFORMIXSERVER is ol_informix1170 (This is from the database that was installed with the informix install, don't know if this must be changed? and if so to what?)
Ran the script you mentioned, result :
INFORMIXDIR=/usr/informix
INFORMIXSERVER=ol_informix1170
INFORMIXSQLHOSTS=/usr/informix/etc/sqlhosts
LANG=en_US.UTF-8
ONCONFIG=onconfig
I noticed I had set the language to UK, which made the Locales en_gb instead if en_us, so tried changing that in my .profile, which did not help, so also tried changing the language to US and the locales to en_us, but this made no difference.
As for what you said about the sqldemo script and the already installed db, It is fine if that db is removed as this is just a test VB box for me to learn on.
Could the $INFORMIXSERVER set as ol_informix1170 be the problem?
Thank you once again for the help.
Neill
Update 2
Thanks again for the response.
A few things to note.
The dbenv results I posted is all that shows which i assume/presume (uh-oh) means that the other environment variables are not set. Which of the environment variables you posted are absolutely necessary for it to work?
As above, Where would I find the terminfo file, or does this need to be created?
As above, the SQLEXEC variable... where would I find sqlrm? I can somewhat remember from the documents I have read I think it should be $INFORMIXDIR/lib? but I only have an esql directory. Is this correct.
Barring that something in the first 3 above is not causing more problems, when trying your suggestion of DEMOPATH=en_us/0333 sqldemo stores2t -log I receive the following error:
Sorry, cannot read the mkstores3 program required to build the demonstration database. Check the /etc subdirectory of INFORMIXDIR (/usr/informix).
Checking /usr/informix/etc shows indeed that there is no mkstores3 file.
Attempting your further note of isqldemo, I get the following error:
/usr/informix/bin/isqldemo: line 58: /usr/informix/demo/sql/en_us/e01c/isqldemo: No such file or directory.
I guess this makes perfect sense as there is no e01c directory, just the 0333 directory.
Right now anything you can tell me would indeed be a consolation because my newb-ness to generally Linux and definately Informix is a big factor. Interesting that this bug has been around for so long. I guess way more experienced folk than I figured out how solve it on their own, or just never bothered with the sqldemo.
I guess that will teach me to read this:
INFORMIX-4GL by Example
Version 4.1
July 1991
Going to check now if any updated text exists, but would still appreciated more help in solving this problem. Do you think reverting to a previous snapshot before Informix was installed and not opting for the ol_informix1170 database to be included could be a possible solution? I wouldn't really see that it would be, but what do I know.
Many many thanks for your continued time and effort.
Regards,
Neill
Update 3
So I see indeed the document I was reading is ancient. I have found an updated one (2002) which uses a different script (dbaccessdemo7).
I tried running that, have run into an error, but tomorrow is another day.
For now I am going to mark this as solved because of the bug detected and resolved. I am not going to put more time and effort into sqldemo.
Thank you so much, and if I struggle with dbaccessdemo 7, I will post a new question.
Regards,
Neill
The sqldemo script won't create a new server; it may clobber your existing database (a single server may house multiple databases; indeed, there are 4 sys* databases created when a server is initialized) but it won't harm your server otherwise.
Probable cause of the error
The normal problem with invalid locales is that you've not set $INFORMIXDIR. You need $INFORMIXDIR set unless /usr/informix is (a symlink to) the correct location. You also need $INFORMIXSERVER set, and you usually need $INFORMIXDIR/bin on $PATH. Strictly, $INFORMIXSERVER is the only mandatory variable; in practice, you worry about the other two too.
The $INFORMIXDIR setting is used to locate the locale information (which is found in $INFORMIXDIR/gls) and the message files (which are found in $INFORMIXDIR/msg).
Note that CP1252 is a Windows code page. Normally on Unix, you'd either not set CLIENT_LOCALE or DB_LOCALE, or you could set them to:
export CLIENT_LOCALE=en_us.8859-1
export DB_LOCALE=en_us.8859-1
or you can choose another more appropriate (to you) locale. The 8859-15 locale includes the Euro symbol, for example, or the utf-8 locale dictates UTF-8 in the database. But, for initial debugging, stick with the 8859-1 locale, aka 819 or 0333 (all based on the IBM CCSID). If it doesn't work with 8859-1, then we have one set of problems; if it works with 8859-1 but not some other codeset or locale, then we have a different set of problems.
Follow-up info if the solution above fails
If that isn't the trouble, then I'll ask for some more details — notably, your Informix environment as reported by the dbenv script below:
: "#(#)$Id: dbenv.sh,v 2.11 2007/09/02 00:18:58 jleffler Exp $"
#
# Printout INFORMIX database environment
informix1="DB[^=]|DELIMIDENT=|SQL|ONCONFIG|TBCONFIG|INFOR"
informix2="ARC_|CLIENT_LOCALE=|GL_|GLS8BITSYS|CC8BITLEVEL|ESQL|FET_BUF_SIZE="
informix3="INF_ROLE_SEP=|NODEFDAC=|ONCONFIG|OPTCOMPIND|PDQ|PSORT"
informix4="PLCONFIG|SERVER_LOCALE|FGL|C4GL|NE_"
informix5="TCL_LIBRARY|TK_LIBRARY"
informix="$informix1|$informix2|$informix3|$informix4|$informix5"
system="COLLCHAR=|LANG=|LC_|LD_LIBRARY_PATH(_64)?=|PATH=|SHLIB_PATH="
jlss="IXD(32|64)?="
env |
egrep "^($informix|$system|$jlss)" |
sort
It's an old script; that's why the shebang is missing.
Second set of diagnosis
I was hoping for the complete output of the dbenv script; it is surprising how often something shows up. However, given what you've said, it is likely to be OK.
The INFORMIXSERVER setting sounds fine.
I'm struck by the LANG=en_US.UTF-8 setting; Informix does pay attention to $LANG and the $LC_* variables (that's why dbenv prints those out). That may be a factor in the problem. However, I would have expected CLIENT_LOCALE and SERVER_LOCALE to deal with that if it was the problem. Also, on my Mac, I have LANG=en_US.UTF-8 and yet I can connect to (8859-1) databases OK.
This is beginning to look like an install problem...or sqldemo problem...
I transitioned from a Mac to a RHEL 5 (archaic) x86/64 machine, and tried running sqldemo over there:
$ dbenv
DBDATE=Y4MD-
DBEDIT=vim
INFORMIXDIR=/work4/informix/tools-7.50.FC4
INFORMIXSERVER=toru_31
INFORMIXSQLHOSTS=/work4/informix/ids-11.70.FC4/etc/sqlhosts
INFORMIXTERM=terminfo
IXD64=/work4/informix/ids-11.70.FC4
IXD=/work4/informix/tools-7.50.FC4
IXH=/work4/informix/ids-11.70.FC4/etc/sqlhosts
IXO=/work4/informix/ids-11.70.FC4/etc/onconfig.toru_31
IXS=toru_31
LANG=en_US.UTF-8
LD_LIBRARY_PATH=/lib64:/usr/lib64:/work4/informix/tools-7.50.FC4/lib:/work4/informix/tools-7.50.FC4/lib/tools:/work4/informix/tools-7.50.FC4/lib/esql:/work4/informix/ids-11.70.FC4/lib:/work4/informix/ids-11.70.FC4/lib/esql:/work4/informix/ids-11.70.FC4/lib/cli
ONCONFIG=onconfig.toru_31
PATH=/work4/informix/tools-7.50.FC4/bin:.:/work4/jleffler/bin:/u/jleffler/bin:/work4/informix/ids-11.70.FC4/bin:/u/jleffler/linux/x86_64/bin:/work4/informix/11.70.FC1:/usr/atria/bin:/work4/jleffler/perl/v5.12.1/bin:/usr/bin:/bin:/usr/X11R6/bin:/atria_release/cm_dist/vobs/imitools/bin:/opt/rational/clearcase/bin:/opt/rational/clearquest/bin
SQLCMDLOG=/work4/jleffler/.sqlcmdlog
SQLEXEC=sqlrm
TERMINFO=/work4/jleffler/terminfo
TERM=xterm-color
$ sqldemo st2 -log
Invalid Locales set !!
$
Oh yeah? No; my locales are fine, thank you!
Well, so be it...I can reproduce your problem! That's step 1. Step 2 is to look at the expletive deleted script.
PRODUCT=sql
DEMOFILE=sqldemo
DEFLANG=en_US.8859-1
INFORMIXDIR=${INFORMIXDIR:=/usr/informix}
INFENV=$INFORMIXDIR/bin/infenv
CONVLOC=$INFORMIXDIR/bin/convloc
if [ $# -gt 0 -a "X$1" = "X-e" ] ; then
LOCALE=$DEFLANG # -e means en_US.8859-1 required
shift
else
LOCALE=`$INFENV DBLANG` # get DBLANG value
if [ "x${LOCALE}" = "x" ]; then
LOCALE=`$INFENV CLIENT_LOCALE` # try CLIENT_LOCALE instead
if [ "x${LOCALE}" = "x" ]; then
LOCALE=`$INFENV DB_LOCALE` # finally default to DB_LOCALE
fi
fi
fi
if [ "x${LOCALE}" = "x" ]; then
LOCALE=$DEFLANG # finally default to DB_LOCALE
fi
export LOCALE
if [ "x${DEMOPATH}" = "x" ]; then
echo "Invalid Locales set !!"
else
exec $INFORMIXDIR/demo/$PRODUCT/$DEMOPATH/$DEMOFILE $*
fi
exit $?
Note that test for ${DEMOPATH}; note that DEMOPATH is not set in the script. So, we've got to get it set. What to? Well, ls $INFORMIXDIR/demo/sql shows that there are various locale-specific sub-directories (en_us,
ja_jp,
ko_kr,
th_th,
zh_cn,
zh_tw) and under the en_us directory there's 0333 (only).
Please run:
DEMOPATH=en_us/0333 sqldemo stores2t -log
This more or less worked for me — I believe it would work for you. I have a slightly unusual setup in that I have just I4GL (p-code and c-code) and ISQL in the $INFORMIXDIR; the server is run out of a different directory. This means I don't have server utility programs like dbload (specifically) in $INFORMIXDIR/bin. When the sqldemo script tried to load the data with dbload, therefore, it failed for me. It would work for you because you have all the Informix software in a single directory. To add insult to injury, it runs the dbload program by explicit path, so I can't futz my PATH to make it available.
This should get you going. I have a bug to report...it is CQ idsdb00244894.
I'm sorry that you ran into so much trouble. You shouldn't have done so.
On my wiki implemented by the MediaWiki interface, I am receiving a Failed to Parse (Unknown Error) for the LaTeX in the page. I checked the LocalSettings.php file, and I have set the proper variable($wgUseTeX) to true.
If it helps, the error message before this was a Failed to Parse(Missing texvc executable), but I "fixed" it to the best of my knowledge by running "make" inside the math directory and installing the texvc executable there. I tested texvc and it works on the commandline.
Could there be anything that I am missing?
aptitude install ocaml
cd /math
make clean
make
Found the answer here: MediaWiki Forums LaTeX Error
It seems that I had to clear the math directory except for the original files, rebuild ocaml, and finally rebuild texvc.
Adding this to LocalSettings solved it for me. Just needed to make the directory calls accurate. Tedious.
$wgUseTeX = true;
$wgUploadDirectory = "{$IP}/images";
$wgUploadPath = "{$wgScriptPath}/images";
$wgMathPath = "{$wgUploadPath}/math";
$wgMathDirectory = "{$wgUploadDirectory}/math";
$wgTmpDirectory = "{$wgUploadDirectory}/tmp";
$wgTexvc = "{$IP}/math/texvc";
Have a look at the README file, paying particular attention to "Ensure that the temporary and math directories exist and can be written to by the user account the web server runs under; if you don't control the server, you may have to make them world-writable."
If that does not help, edit render.ml, comment out the block that says "Commenting this block out will aid in debugging", and re-run make. This will leave all temporary files (including the TeX log) so that you can hopefully see what went wrong.