Using Mosek with Drake on Deepnote - drake

ValueError: "MosekSolver cannot Solve because MosekSolver::available() is false, i.e., MosekSolver has not been compiled as part of this binary. Refer to the MosekSolver class overview documentation for how to compile it."
Hi, I got the above error when trying to use the Mosek solver in Drake. It is not clear to me how to enable Mosek in Deepnote with Drake. Do I need to include something in the Dockerfile or the init file? Any tips would be appreciated.
Links I looked at:
https://drake.mit.edu/pydrake/pydrake.solvers.mosek.html
https://drake.mit.edu/bazel.html#mosek

Mosek+Drake does work on Deepnote. The workflow is like this:
Obtain a Mosek license file (from the Mosek website), and upload it to Deepnote.
Set an environment variable to tell Drake where to find the license file. For instance, you can add the following at the top of your notebook:
import os
os["MOSEKLM_LICENSE_FILE"] = "mosek.lic"
Now MosekSolver.available() should be True, and Mosek will even be chosen as the default preferred solver for if you simply call Solve(prog).
Note: Please be very careful not to share the Deepnote notebook with your mosek.lic uploaded.

Related

nltk.download('punkt') giving output as false

When I trying to install nltk and download the file punket using nltk.download('punkt').
I am getting the following errors. Have tried many alternative codes and changing networks.
error
Please help with this error.
Post applying :-
= df['num_words'] = df['text'].apply(lambda x:len(nltk.word_tokenize(x)))
I am gettring the error:-
**Resource punkt not found.
Please use the NLTK Downloader to obtain the resource:
import nltk
nltk.download('punkt')
For more information see: https://www.nltk.org/data.html
Attempted to load tokenizers/punkt/english.pickle**
I tried some alternative codes like
import nltk
import ssl
try:
_create_unverified_https_context = ssl._create_unverified_context
except AttributeError:
pass
else:
ssl._create_default_https_context = _create_unverified_https_context
nltk.download()
Also tried changing the networks as at some places I found it is saying server issue.
Try to launch the jupyter notebooks session as administrator (open the command or anaconda prompt as administrator).
The last option would be to download the corpus manually. You may find this, helpful in your case.

How do I access parsing of .nimble files from the nimble package?

Nim as a language provides .nimble files to describe its packages (example of a .nimble file). I know that the file is parsed by the nimble package and CLI-tool, as they need the information inside the .nimble file for their tasks.
I want basically all the information in there, dependencies, author, license, description, version, all of it. So in order to not do the same work twice and potentially run into issues should the format change, I would like to use the nimble package itself to parse the .nimble file for me.
I know the correct proc for it, which is getPkgInfoFromFile, but I can't seem to access it with import nimble/nimblepkg/packageparser.
Whenever I use that line I receive an error that there is no such file.
What am I doing wrong?
Further: getPkgInfoFromFile requires an Options instance that it generates when parsing a CLI command. I don't have a CLI command, so I'm not generating such an instance, can I use the proc somehow without one?
Thanks to ringabout I came to the correct solution, but first to the question.
Question 1: How do I access the proc in the first place?
You can access nimble like a package, but the import is not import nimble/nimblepkg/packageparser it is directly import nimblepkg/packageparser.
This requires you to have both nimble' installed as a library as well as the compiler` installed as a library.
So you'll have to install those first:
nimble install nimble
nimble install nim # Originally this was called "compiler", but was renamed to "nim"
Ignore any warnings if they pop up.
Now you can compile the following dummy-example:
#dummy.nim
import nimblepkg/packageparser
echo "Pointer to packageparser proc: ", packageparser.getPkgInfoFromFile.repr
with: nimble -d:ssl --mm:refc -r build (-d:ssl is required for nimble's HTTP-client and --mm:refc is required as nimble appears to not work with orc)
Question 2: Can I run the getPkgInfoFromFile without an Options instance?
Yes-ish. You still need one, but it doesn't have to be a "real" one, you can just instantiate one yourself on the fly.
import nimblepkg/[options, packageinfotypes, packageparser]
proc generateDummyOptions(): Options =
result = initOptions()
result.setNimBin()
result.setNimbleDir()
proc parseNimbleFile*(nimblePath: string): PackageInfo =
let options = generateDummyOptions()
result = getPkgInfoFromFile(nimblePath.NimbleFile, options)

import python functions into serversless

I h got an issue with the following serverless config.
this is my handler and the files/folders structure.
the issue is that after uploading my project to AWS when I test my lambda I got an error as follows:
lambda execution fails: "errorMessage": "Unable to import module 'app_monitor': No module named 'monitoring'"
{
"errorMessage": "Unable to import module 'src/app_monitor': No module named 'monitoring'",
"errorType": "Runtime.ImportModuleError",
"requestId": "bca3f67d-815f-452b-a2a6-c713ad2c6baa",
"stackTrace": []
}
have you got any clue how can I add this into serverless config.?
First, a quick tip on troubleshooting: When I ran into such issues it was helpful to go to the AWS console, look at the lambda function, and see what the uploaded file structure looks like on that end. Is the monitoring folder there?
Moreover, in order to specify how a specific function is packaged, you have to explicitly state that you want it to be individually packaged and not follow the general rules of the project as a whole.
You should try to add:
app_monitoring:
package:
individually: true
patterns:
- 'src/**'
More documentation on packaging configuration here
You may also have better luck with explicitly stating the patterns you need, I know I've had issues with globs in the past. So for example you can try:
patterns:
- 'src/app_monitoring.py'
- 'src/monitoring/get_lb.py'

Biopython SeqIO error: local variable 'qual' referenced before assignment

I send some samples for Sanger sequencing to a commercial facility. I'm able to read the files they send using the command
from Bio import SeqIO
from Bio import Seq
rec = SeqIO.read("isolation-round4/3dr23_Forward.ab1",'abi-trim').seq
But recently, due to a move, we had to send the samples elsewhere for sequencing. Now, if I try to run the same command on the output I get an error:
UnboundLocalError: local variable 'qual' referenced before assignment in
File "C:\Users\Anaconda3\lib\site-packages\Bio\SeqIO\AbiIO.py", line 462, in AbiIterator letter_annotations={"phred_quality": qual}
I would appreciate any help in dealing with this. Here are two files, one that works and one that does not, if you would like to have a look.
Thanks in advance for your help!
Bug should have already been fixed in Biopython 1.77
Update: See https://github.com/biopython/biopython/issues/3221 - turned out to be a new unexpected configuration of the ABI software producing files with no quality scores.

Using TensorFlow Audio Recognition Model on iOS

I'm trying to use the TensorFlow audio recognition model (my_frozen_graph.pb, generated here: https://www.tensorflow.org/tutorials/audio_recognition) on iOS.
But the iOS code NSString* network_path = FilePathForResourceName(#"my_frozen_graph", #"pb"); in the TensorFlow Mobile's tf_simple_example project outputs this error message: Could not create TensorFlow Graph: Not found: Op type not registered 'DecodeWav'.
Anyone knows how I can fix this? Thanks!
I believe you are using the pre-build Tensorflow from Cocapods? It probably does not have that op type, so you should build it yourself from latest source.
From documentation:
While Cocapods is the quickest and easiest way of getting started, you
sometimes need more flexibility to determine which parts of TensorFlow
your app should be shipped with. For such cases, you can build the iOS
libraries from the sources. This guide contains detailed instructions
on how to do that.
This might also be helpful: [iOS] Add optional Selective Registration of Ops #14421
Optimization
The build_all_ios.sh script can take optional
command-line arguments to selectively register only for the operators
used in your graph.
tensorflow/contrib/makefile/build_all_ios.sh -a arm64 -g $HOME/graphs/inception/tensorflow_inception_graph.pb
Please note this
is an aggresive optimization of the operators and the resulting
library may not work with other graphs but will reduce the size of the
final library.
After the build is done you can check /tensorflow/tensorflow/core/framework/ops_to_register.h for operations that were registered. (autogenerated during build with -g flag)
Some progress: having realized the unregistered DecodeWav error is similar to the old familiar DecodeJpeg issue (#2883), I ran strip_unused on the pb as follows:
bazel-bin/tensorflow/python/tools/strip_unused \
--input_graph=/tf_files/speech_commands_graph.pb \
--output_graph=/tf_files/stripped_speech_commands_graph.pb \
--input_node_names=wav_data,decoded_sample_data \
--output_node_names=labels_softmax \
--input_binary=true
It does get rid of the DecodeWav op in the resulting graph. But running the new stripped graph on iOS now gives me an Op type not registered 'AudioSpectrogram' error.
Also there's no object file audio*.o generated after build_all_ios.sh is done, although AudioSpectrogramOp is specified in tensorflow/core/framework/ops_to_register.h:
Jeffs-MacBook-Pro:tensorflow-1.4.0 zero2one$ find . -name decode*.o
./tensorflow/contrib/makefile/gen/obj/ios_ARM64/tensorflow/core/kernels/decode_bmp_op.o
./tensorflow/contrib/makefile/gen/obj/ios_ARM64/tensorflow/core/kernels/decode_wav_op.o
./tensorflow/contrib/makefile/gen/obj/ios_ARMV7/tensorflow/core/kernels/decode_bmp_op.o
./tensorflow/contrib/makefile/gen/obj/ios_ARMV7/tensorflow/core/kernels/decode_wav_op.o
./tensorflow/contrib/makefile/gen/obj/ios_ARMV7S/tensorflow/core/kernels/decode_bmp_op.o
./tensorflow/contrib/makefile/gen/obj/ios_ARMV7S/tensorflow/core/kernels/decode_wav_op.o
./tensorflow/contrib/makefile/gen/obj/ios_I386/tensorflow/core/kernels/decode_bmp_op.o
./tensorflow/contrib/makefile/gen/obj/ios_I386/tensorflow/core/kernels/decode_wav_op.o
./tensorflow/contrib/makefile/gen/obj/ios_X86_64/tensorflow/core/kernels/decode_bmp_op.o
./tensorflow/contrib/makefile/gen/obj/ios_X86_64/tensorflow/core/kernels/decode_wav_op.o
Jeffs-MacBook-Pro:tensorflow-1.4.0 zero2one$ find . -name audio*_op.o
Jeffs-MacBook-Pro:tensorflow-1.4.0 zero2one$
Just verified that Pete's fix (https://github.com/tensorflow/tensorflow/issues/15921) is good:
add this line tensorflow/core/ops/audio_ops.cc to the file tensorflow/contrib/makefile/tf_op_files.txt and run tensorflow/contrib/makefile/build_all_ios.sh again (compile_ios_tensorflow.sh "-O3" itself used to work for me after adding a line to the tf_op_files.txt, but not anymore with TF 1.4).
Also, use the original model file, don't use the stripped version. Some note was added in the link above.

Resources