I'm using Dbpedia in my project and I wanted to create a local sparql end point as the online one is not reliable. I downloaded data dumps (large NT files) and decided to use Jena TDB. Using the NetBeans IDE am using an input stream to read in the source NT file and then using the following line of code to load the NT file into a datasetGraph:
TDBLoader.load(indexingDataset, inputs, true);
I let it run for about 5 hrs now and it still isnt done. Whilst doing this everything on my laptop seems to slow down probably because of it taking all my physical memory space. Is there a faster way to do this???
The documentation says to use tdbloader2 but its only available for linux while am using windows. Would be really helpful if anyone could tel me how to use this tool in windows using cygwin. Please take into consideration I have never really used Cygwin in windows.
The latest release of TDB has two command line utilities for bulk loading: tdbloader and tdbloader2. The first is pure Java and it runs on Windows as well as on any machine with a JVM. The second is a mix of Java and UNIX shell script (in particular it uses UNIX sort). It runs on Linux, I am not sure it runs on Cygwin. I suggest you use tdbloader on a 64-bit machine with as much RAM as you can find. :-)
The latest release of TDB is available here:
http://www.apache.org/dist/incubator/jena/jena-tdb-0.9.0-incubating/jena-tdb-0.9.0-incubating-distribution.zip
The development version of TDB has an additional bulk loader command: tdbloader3. This is a pure Java version of tdbloader2. Instead of using UNIX sort (which works only on text files) we used a pure Java external sort with binary files. For more details on tdbloader3, search for the JENA-117 issue.
You can find a SNAPSHOT of TDB in the Apache snapshots repository, you are warned, that has not been released yet.
For the more adventurous there is also tdbloader4 which is not included in Apache Jena and it is to be considered an experimental prototype.
tdbloader4 builds TDB indexes (i.e. B+Tree indexes) using MapReduce (this is stretching a little bit the MapReduce model, but it works).
You can find tdbloader4 here: https://github.com/castagna/tdbloader4
To conclude, my advise to you, on Windows, is: download the latest official release of TDB and use tdbloader with a 64-bit machine with a lot of RAM. If you do not have one, use an m1.xlarge EC2 instance (i.e. 15 GB of RAM) (or equivalent).
For more help, I invite you to join the official jena-users#incubator.apache.org mailing list where, I am sure, you'll have better and faster support.
Related
I come across the following link - https://github.com/felixrieseberg/windows95
Running windows 95 as Electron APP
Is it possible to create any existing application as portable or cross-platform using electron like above?
Can we use Windows 7 or 10 like above without Oracle Virtual Box or VM in an existing Windows 8 machine ?
Need more details on how it is possible to bring an application via electron
Thanks.
If I'm hearing you right, you're wondering if you can put a whole OS into a JS VM running in Electron - and then run your app on top of that.
Please don't do that. Electron is a fantastic cross-platform solution, but my project windows95 was mostly an educational experiment showcasing how powerful modern JavaScript and Electron can be.
This project uses another project called v86, which is capable of virtualising a x86 platform and display its output on a HTML element. Electron is browser-based so that is why this works.
But there are some limitations: It virtualises the x86 platform, which is pretty old by now, and when you have a look at the specification, it says it is at a Pentium-1 state. The demo links on the project's GitHub ReadMe also do not contain any "modern" OS because v86 does not support SSE(2). And since Microsoft requires its users to have a SSE2-compatible CPU* it is not possible to run anything like Windows 7 or newer (Linux may work, anyway).
OSses which support this kind of (deprecated) hardware, may work. As you can see from the "Compatibility" section of the project's ReadMe, some Linux versions and Windows XP "work" but older software, like OS/2, and even newer stuff, like Android, don't (really). So you can't say that everything will work as expected because the execution environment is really limited.
Now to answer your first point, no, I don't think that you can wrap every existing app into an Electron application and run it on new hardware. It may work with old software (which do not require SSE, if you use v86, and that would require a lot of hackery on the v86 side because it is designed to run OSses), but don't quote me on that.
* The section in question reads (at the time of writing) like this:
A Stop error occurs on computers that don't support Streaming Single Instructions Multiple Data (SIMD) Extensions 2 (SSE2).
Upgrade your machines with a processor that supports SSE2 or virtualize those machines.
(emphasis mine, source: https://support.microsoft.com/en-us/help/4088875/windows-7-update-kb4088875)
How can I build and compile my own Lua files on Windows? And make them executable.
I am reading Beginning Lua programming, and I have Windows 7 and MacOS Lion both installed. I am having the hard time to follow the instructions. They do not work for me.
On MacOS I open the terminal and put these in:
export LUA_DIR=/usr/local/lib/lua/5.1
mkdir -p /usr/local/lib/lua/5.1 (it tells me, mkdir: illegal option) and I can not follow from here
SET LUA_DIR=”c:\program files\lua\5.1”
As for Windows I do this according to the book.
This what I see in my shell c:\Users\bd>
mkdir "c:\program files\utility" and it tells me access is denied
I have tried to right click on this folder and check off read only, but it does not work.
Any clues would be appreciated, this part has been really confusing for me.
To package your Lua files into an executable on Windows you have several options. There is srlua, there is wxLuaFreeze from wxLua (available as a binary for Windows), and there are more options in this SO answer.
Essentially, the main two options are: (1) append your Lua code to a precompiled exe file, such that it will be loaded and executed when that exe file is run, and (2) convert your Lua code into real executable by compiling it to bytecode, then to C, and then to your target platform.
As to your MacOS issue, mkdir -p means that mkdir is asked to create intermediate directories (for example, you asked to create /a/b/c, it will also create /a/b if those don't exist). As you don't say which version of MacOS you run, it's difficult to provide more detailed answer.
For now the standard distribution of Lua does not compile a script to native executable code; it execute your scripts by first compiling it to bytecode, then by interpreting the bytecode with a reasonnably fast static interpret (this also means that it is easily portable across native or virtual systems, and very resistant to attacks (that could be targetting bugs in the native compiler itself).
Also Lua still does not feature a runtime JIT compiler like Java and .Net: Lua still does not features a VM to produce a safe sandbox.
There exists Lua packages that convert your bytecode (or directly a source script) to a C source that can be used to convert a Lua library into native mode via the same C compiler used to compile the Lua engine itself (this is how the builtin libraries are produced, though they are slightly optimized manually in some time-critical parts).
However it is possible to compile Lua to a javascript source, and run it with fast performance using Javascript, because today's Javascript interprets do have good performance with their implemented VM featuring a JIT compiler for their own bytecodes.
It is also possible by converting it the Lua bytecode to a .Net or Java source that can then be executed directly from Lua (for that you need a version of Lua that has been ported to .Net or Java or Javascript, something that is not so complicate than developing in C/C++ directly a VM with a JIT compiler (a moderately complex part is the bytecode verifier, but the really complex part is the memory manager its garbage collector and its sandbox so that your Lua script will be fully isolated from the Lua engine itself for itw own memory, but the most complex part if the runtime optimizer and collection of profiling statistics: this has been done in the modern VMs for Java, .Net, Javascript, PHP/Zend, Python, Perl...).
I dont know which other language VM would offer the best performance to port Lua and implement on it a compiler to their own bytecode running at near native speed in their VM. But my own small experience with programs (in a much simpler language) self-generating a bytecode that they can run themselves, has always shown me Java winning in performance over .Net and Javascript. This is most probably because Java features an profiling-based dynamic code optimizer
(On the opposite the .Net optimizer runs only once during program installation, using some profiling data collected during the installation of the .Net VM itself, or at first instanciation of the script, without really knowing any profiling data collected during execution of the compiled program itself, and based on some cheked assumptions about the platform capabilities).
I also don't if would be faster in PHP, Python or Perl; the comparison with newer Javascript engines was never attempted though. Porting/compiling a Lua program to Javascript is relatively easy because it implements closures relatively easy for the resolution of linkages. Then the generated Javascript will compile to native code with the excellent Javascript's JIT compilers we have today (and never cease to improve in performance, so much that I've seen various appliactions running now faster in Javascript than before when they were written in C++ or plain C; as well the memory footprint has largely been reduced, we no longer have memory leaks, and even if there's a garbage collector, today's Javascript VM have a very efficient one, which is even better than the GC implemented in the native Lua).
But Lua remains useful as it is easy to secure and sandbox and offers various security benefits (but there are security issues in Lua as well for some kinds of applications, where Javascript offers some solutions, notably for side-channel attacks based on variation of time of execution; but these side-channel attacks are very hard to solve and can affect any system, any program, any programming language, and this starts becoming a critical issue because they are now more esily exploitable; the reason of that comes from hardware optimizations that we depend more and more today when we want to maximize the performances). And with Lua you may be more immune to these problems that a sandboxing sofware environment cannot solve alone.
Probably later we'll see a true VM implementation of Lua with a JIT and self-generating code and the possibility to instanciate new sandboxed VMs to run their self-generated code. It will take more time to generate an EXE file for distribution; notably because it generally requires adding also an installer and a distribution manager.
So for now we could imagine distributing Lua applications compiled to the bytecode of another JIT-capable VM: this generated bytecode would be faster than the Lua bytecode, and would then be extremely complex to reverse-engineer to the semantics of Lua because it would require two separate reverse engineering first from the bytecode of the other VM to the bytecode of Lua, both bytecodes loosing some easiy inferable rules and options tested and foll, and then again to sme Lua source
For the OSX terminal issue:
This command should work
export LUA_DIR=/usr/local/lib/lua/5.1
This command will probably give you permission problems:
mkdir -p /usr/local/lib/lua/5.1
You may try this to solve that. You will be prompted for your password:
sudo mkdir -p /usr/local/lib/lua/5.1
This command has nothing to do with OSX and will not work. This is a windows command:
SET LUA_DIR=”c:\program files\lua\5.1”
You have a permissions problem with Windows- try creating your cmd or PowerShell in Administrator mode. C:\Program Files is a protected directory that a regular user account doesn't have permission to write to.
As for the OS X issue, check out the mkdir OS X manual page to make sure you have the command correct.
So, if I understood your question correctly, you are trying to build Lua on Windows.
This is of course possible, but not easy for beginners. I would highly recommend you to use a binary distribution, which is much easier to install, unless you have special requirements.
Here are several Windows distributions :
Lua Binaries (Lua 5.1 and 5.2)
LuaForWindows (Lua 5.1)
LuaDist (Lua 5.2)
I've done some search out there but couldn't find too much really helpful info on it, but could someone try to explain the basic of Java memory maps? Like where/how to use it, it's purpose, and maybe some syntax examples (inputs/outputs types)? I'm taking a Java test soon and this could be one of the topics, but through all of my tutorials Jmap has not come up. Thanks in advance
Edit: I'm referring to the tool: jmap
I would read the man page you have referenced.
jmap prints shared object memory maps or heap memory details of a given process or core file or a remote debug server.
NOTE: This utility is unsupported and may or may not be available in future versions of the JDK. In Windows Systems where dbgeng.dll is not present, 'Debugging Tools For Windows' needs to be installed to have these tools working. Also, PATH environment variable should contain the location of jvm.dll used by the target process or the location from which the Crash Dump file was produced.
http://docs.oracle.com/javase/7/docs/technotes/tools/share/jmap.html
Its not a tool to be played with lightly. You need a good profiler which can read it output as jhat is only useful for trivial programs. (YourKit works just fine for 1+ GB heaps)
Is it Possible to create bootable (Applications for MBR )application using Delphi or Pascal (I know we cant use vcl , RTL and other stuffs because they depend on OS), but can i use at least Readln and writeln.
If it is true !!! Can we run the program under other OS.
but i know that PE (windows) and ELF (Linux ) formats are different. but at least with some small modification can i do it.
It's worth saying that PE is a very diverse format than ELF.
Not only a few bytes to modify... the whole layout and library access is diverse, and binding is totally diverse.
In order to boot Delphi application in console mode, you can put a small DOS system (take a look at FreeDOS, for instance), then run your Delphi application using for instance DWPL. DWPL allows to run native 32-bit protected mode DOS programs with Delphi 5-7 using the WDOSX DOS extender as the core. I used this in some old hardware with a network adapter, and it worked like a charm. If you are interested in it, I could post some updated code of DWPL.
For such targets, you should take a look at Free Pascal. By nature, you can customize it to whatever target you want. There is even diverse draft Operating Systems written using FPC. See for instance Toro or ClassiOS - the latest uses Delphi executables as source.
You can see the boot code of Toro from here, and a "main program" source code created with it.
But for direct booting applications, booting is not so difficult. The real problem is the hardware layer.
The BIOS gives very little access to it.
Just for the network layer, you'll have to take a look at EtherBoot sites and such to get some low-level network access... but it could be very time consuming to rewrite all those drivers by hand!
In short: all those "pure pascal" OS are only theoretical, running a console and some low-performance network (emulating a poor network adapter like NE2000 or such). So those "pascal" OS are only proof of concept. FAR away from a working solution! But very nice technological challenge, in all cases, very inspiring.
Why reinvent the wheel? If you want a light and fast system, use a custom Linux kernel.
Then use CrossKylix to compile your Delphi application (with no User Interface) into Linux, or even better Free Pascal.
You don't really place "applications" in the MBR.
The entire size of an MBR is 512 bytes, of which you can only use 446 for code.
Good luck creating something useful in that if you don't even have an OS to delegate functionality to yet. Basically all that you can do in the MBR is place code to start a boot loader.
Here's a page with disassembly of an MBR:
http://www.dewassoc.com/kbase/hard_drives/master_boot_record.htm
Why must you write the boot loader?
You could use a ready-made bootloader like GRUB and chainload your PE executable, from it.
Of course, this is very ancient and hairy stuff, but in the good old days, people did this win PE format executables, and a DOS Extender.
For something a little more this-century, why not make your own bootable REACTOS disk, and add your own PE executable written in Delphi to handle the "user shell"?
You could also (but this would require licensing) use the Windows PXE. I think that projects like BartPE probably fall on the gray side of legal, or are at least, unlicensed. Thus, a completely MS-free solution (reactos) for a completely self-contained kiosk PC, with ReactOS, might be more what you are looking for.
Can you write your own operating system? your own UI layer? your own video device drivers? I didn't think so. So use DOS and TurboPascal, or ReactOS and a PE win executable. Or you can use FreePascal and just build your app on a very lightweight portable Linux kernel and root filesystem.
I'm using flex (lexical analyzer, not Adobe Flex) on a project. However, I want to be able to compile on Windows platforms as well, but the Windows version's newest version is only 2.5.4a, so it won't compile my file for version 2.5.35. And no, I can't downgrade to the highest supported Windows version.
Anyone know about plans to upgrade the windows version, or have a suggestion to compile on windows anyway?
You can ask on the mailing list, or get involved in the Flex project yourself. I think the code-base for Flex has remained static for a while, but I don't know who maintains the Windows port. In the interim...
I would recommend including the produced source in your project.
Generate the lexer on a Linux system to produce your lex.c/lex.h files (or whatever)
Include those files in your Win32 C source before you build
If you don't have direct access to a Linux system, a virtual machine might be a good idea. The Flex source should be complaint to some C standard that builds on Windows, but most of the POSIX differences can be altered to use Win32 API fairly easily.
Maybe distribute as:
/src/source_files.c
/src/lex.l
/src/win32_lex/lex.c
This way systems with a modern flex can generate the source from the lex file, and Windows systems compiling the source can use the complementary pre-processed C files.
Short of using some user-space POSIX (Cygwin or whatever).
A little bit of tweaking required, but isn't that portability for you!
Windows builds of flex 2.5.35 do exist, but unfortunately they are not self contained. You can download the MINGW build here, and the Cygwin build here; see also another stackoverflow question. Each build requires that its respective (MINGW or Cygwin) kernel be installed.