Is this information contained in these chunks of the Beam file? Or is there another way?
References: Erlang beam_lab
For a beam file compiled from Erlang, you can get the number of lines like this:
{ok,{_ModuleName,[{debug_info,{debug_info_v1,erl_abstract_code, DebugInfoData}}]}} =
beam_lib:chunks("my_module", [debug_info]).
{ok, Defs} = erl_abstract_code:debug_info(erlang_v1, x, DebugInfoData, []).
{eof, {Lines, _}} = lists:last(Defs).
Replace "my_module" with the file name of the beam file without the .beam extension.
Related
I am learning the Lua IO library. I'm having trouble with io.write(). In Programming Design in Lua, there is a piece of code that iterates through the file line by line and precedes each line with a serial number.
This is the file I`m working on:
test file: "iotest.txt"
This is my code
io.input("iotest.txt")
-- io.output("iotest.txt")
local count = 0
for line in io.lines() do
count=count+1
io.write(string.format("%6d ",count), line, "\n")
end
This is the result of the terminal display, but this result cannot be written to the file, whether I add IO. Output (" iotest.txt ") or not.
the results in terminal
This is the result of file, we can see there is no change
The result after code running
Just add io.flush() after your write operations to save the data to the file.
io.input("iotest.txt")
io.output("iotestout.txt")
local count = 0
for line in io.lines() do
count=count+1
io.write(string.format("%6d ",count), line, "\n")
end
io.flush()
io.close()
Refer to Lua 5.4 Reference Manual : 6.8 - Input and Output Facilities
io.flush() will save any written data to the output file which you set with io.output
See koyaanisqatsi's answer for the optional use of file handles. This becomes especially useful if you're working on multiple files at a time and gives you more control on how to interact with the file.
That said you should also have different files for input and output. You'll agree that it doesn't make sense to read and write from and to the same file alternatingly.
For writing to a file you need a file handle.
This handle comes from: io.open()
See: https://www.lua.org/manual/5.4/manual.html#6.8
A file handle has methods that acts on self.
Thats the function after the : at file handle.
So io.write() puts out on stdout and file:write() in a file.
Example function that can dump a defined function to a file...
fdump=function(func,path)
assert(type(func)=="function")
assert(type(path)=="string")
-- Get the file handle (file)
local file,err = io.open(path, "wb")
assert(file, err)
local chunk = string.dump(func,true)
file:write(chunk)
file:flush()
file:close()
return 'DONE'
end
Here are the methods, taken from io.stdin
close = function: 0x566032b0
seek = function: 0x566045f0
flush = function: 0x56603d10
setvbuf = function: 0x56604240
write = function: 0x56603e70
lines = function: 0x566040c0
read = function: 0x56603c90
This makes it able to use it directly like...
( Lua console: lua -i )
> do io.stdout:write('Input: ') local result=io.stdin:read() return result end
Input: d
d
You are trying to open the same file for reading and writing at the same time. You cannot do that.
There are two possible solutions:
Read from file X, iterate through it and write the result to another file Y.
Read the complete file X into memory, close file X, then delete file X, open the same filename for writing and write to it while iterating through the original file (in memory).
Otherwise, your approach is correct although file operations in Lua are more often done using io.open() and file handles instead of io.write() and io.read().
I am trying to load yaml from files created by
entries = bag.from_sequence([{1:2}, {3:4}])
yamls = entries.map(yaml.dump)
yamls.to_textfiles(r'\*.yaml.gz')
with
yamls = bag.read_test(r'\*.yaml.gz', linedelimiter='\n\n)
but it reads files line by line. How to read yamls from files?
UPDATE:
While blocksize=None read_text reads files line by line.
If blocksize is set, you could read compressed files.
How to overcome this? Is uncompressing the files is the only option?
Indeed, linedelimiter is used not for the sense you have in mind, but only for separating the larger blocks. As you say, when you compress with gzip, the file is no longer random-accessible, and blocks cannot be used at all.
It would be possible to pass the linedelimiter into the functions that turn chunks of data into lines (in dask.bag.text, if you are interested).
For now, a workaround could look like this:
yamls = bag.read_test(r'\*.yaml.gz').map_partitions(
lambda x: '\n'.join(x).split(delimiter))
I have to work with some SWI-Prolog code that opens a new stream (which creates a file on the file system) and pours some data in. The generated file is read somewhere else later on in the code.
I would like to replace the file stream with a string stream in Prolog so that no files are created and then read everything that was put in the stream as one big string.
Does SWI-Prolog have string streams? If so, how could I use them to accomplish this task? I would really appreciate it if you could provide a small snippet. Thank you!
SWI-Prolog implements memory mapped files. Here is a snippet from some old code of mine, doing both write/read
%% html2text(+Html, -Text) is det.
%
% convert from html to text
%
html2text(Html, Text) :-
html_clean(Html, HtmlDescription),
new_memory_file(Handle),
open_memory_file(Handle, write, S),
format(S, '<html><head><title>html2text</title></head><body>~s</body></html>', [HtmlDescription]),
close(S),
open_memory_file(Handle, read, R, [free_on_close(true)]),
load_html_file(stream(R), [Xml]),
close(R),
xpath(Xml, body(normalize_space), Text).
Another option is using with_output_to/2 combined with current_output/1:
write_your_output_to_stream(Stream) :-
format(Stream, 'example output\n', []),
format(Stream, 'another line', []).
str_out(Codes) :-
with_output_to(codes(Codes), (
current_output(Stream),
write_your_output_to_stream(Stream)
)).
Usage example:
?- portray_text(true), str_out(C).
C = "example output
another line"
Of course, you can choose between redirecting output to atom, string, list of codes (as per example above), etc., just use the corresponding parameter to with_output_to/2:
with_output_to(atom(Atom), ... )
with_output_to(string(String), ... )
with_output_to(codes(Codes), ... )
with_output_to(chars(Chars), ... )
See with_output_to/2 documentation:
http://www.swi-prolog.org/pldoc/man?predicate=with_output_to/2
Later on, you could use open_string/2, open_codes_stream/2 and similar predicates to open string/list of codes as an input stream to read data.
erlang is a strange language to me, this week i've been playing with multiple languages and I've come here for help often, now I'm on erlang and I'm stuck once again :)
Basically All i'm trying to do is the following but in erlang:
Dim objFSO, objFile, objFolder
Set objFSO = Server.CreateObject("Scripting.FileSystemObject")
Set objFolder = objFSO.GetFolder(currentDirectory))
For Each objFile in objFolder.Files
do something with the file
do something else
do more stuff
Next
The closest i've come is:
-export([main/1]).
main([]) ->
find:files("c:\","*.txt", fun(F) -> {
File, c:c(File)
}end).
clearly, not working and nothing like how i'd need it to be.. but I have tried many ways and read many example but simply cant figure out a solution maybe the language just isnt ment for this kind of stuff?
And this is needed as a escript (erlang script)
It's tough to recommend exactly which method you should use because your "do something" pseudocode is too vague.
There are two main ways one iterates over a list of something in a functional language like Erlang: map and fold.
The big question comes down to this: What do you want to do with the files? Do you want to total something up for the files (ie, total file size or something), or do you want to store some value for each file (ie each file size individually) or do you want to do something to the files and you don't care what the return values for those files are (ie renaming each file)?
I'll give an example of each quickly here using a list of files returned from file:list_dir/1:
{ok, Filenames} = file:list_dir("some_directory"),
Folding
Here we'll total the filesizes of all files in the directory using lists:foldl (as #legoscia mentioned, in this case, filelib:fold_files is probably the better choice)
TotalSize = lists:foldl(fun(Filename,SizeAcc) ->
FileInfo = file:read_file_info("some_directory/" ++ Filename),
FileSize = FileInfo#file_info.size,
SizeAcc + FileSize
end, 0, Filenames).
Mapping
Here, we'll get a list of filenames along with the filesize for each individual file using lists:map. The resultant list will be of the format = [{"somefile.txt",452}, {"anotherfile.exe",564},...]:
FileSizes = lists:map(fun(Filename) ->
FileInfo = file:read_file_info("some_directory/" ++ Filename),
FileSize = FileInfo#file_info.size,
{Filename,FileSize}
end,Filenames).
Foreach (a variant of mapping)
The alternative of just renaming files but not caring about recording any data about the files is to demonstrate the use of lists:foreach, which is generally used exclusively for side-effect programming in which you don't care about the return values, and it works like lists:map, but doesn't return anything useful (it just returns the atom ok):
In this case, I'll show renaming each file by adding a ".old" extension to each filename:
lists:foreach(fun(Filename) ->
OldFile = "some_directory/" ++ Filename,
NewFile = OldFile ++ ".old",
file:rename(OldFile, NewFile),
end,Filenames).
Recursion
Of course, the raw version of all of these - if map, fold, foreach, or list comprehensions (which I didn't cover, but are basically another variant of map with a filter component) are too restricting for whatever reason - you can do things recursively:
do_something_with_files([]) -> ok;
do_something_with_files([CurrentFile|RestOfFiles]) ->
do_something(CurrentFile),
do_something_with_files(RestOfFiles).
There are a lot of ways to do what you need with Erlang, but unlike with procedural languages like VB, you must think a little ahead to what you want to track or do in order to determine how you wish to iterate over your lists, since you're limited by immutable variables in Erlang.
Note: In order to use the #file_info record, you'll need to include the file.hrl file at the top of your module with:
-include_lib("kernel/include/file.hrl").
i wrote a file system indexer recently and here below, i provide you with a module, which can traverse a directory (s) and lets you so what ever you want with the files it finds or the folders it finds. What it does is that it will spawn a new process to handle any inner directory. You will provide two Functional Objects , one that will deal with directories and the other will deal with the files.
%% #doc This module provides the low level APIs for reading, writing,
%% searching, joining and moving within directories.
%% #end
-module(file_scavenger_utilities).
%%% ------- EXPORTS -------------------------------------------------------------------------------
-compile(export_all).
%%% ------- INCLUDES -----------------------------------------------------------------------------
%%% -------- MACROS ------------------------------------------------------------------------------
-define(IS_FOLDER(X),filelib:is_dir(X)).
-define(IS_FILE(X),filelib:is_file(X)).
-define(FAILED_TO_LIST_DIR(X),
error_logger:error_report(["* File Scavenger Utilities Error * ",
{error,"Failed to List Directory"},{directory,X}])).
-define(NOT_DIR(X),
error_logger:error_report(["* File Scavenger Utilities Error * ",
{error,"Not a Directory"},{alleged,X}])).
-define(NOT_FILE(X),
error_logger:error_report(["* File Scavenger Utilities Error * ",
{error,"Not a File"},{alleged,X}])).
%%%--------- TYPES -------------------------------------------------------------------------------
%% #type dir() = string().
%% Must be containing forward slashes, not back slashes. Must not end with a slash
%% after the exact directory.e.g this is wrong: "C:/Program Files/SomeDirectory/"
%% but this is right: "C:/Program Files/SomeDirectory"
%% #type file_path() = string().
%% Must be containing forward slashes, not back slashes.
%% Should include the file extension as well e.g "C:/Program Files/SomeFile.pdf"
%% -----------------------------------------------------------------------------------------------%% #doc Enters a directory and executes the fun ForEachFileFound/2 for each file it finds
%% If it finds a directory, it executes the fun %% ForEachDirFound/2.
%% Both funs above take the parent Dir as the first Argument. Then, it will spawn an
%% erlang process that will spread the found Directory too in the same way as the parent directory
%% was spread. The process of spreading goes on and on until every File (wether its in a nested
%% Directory) is registered by its full path.
%% #end
%%
%% #spec new_user(dir(),funtion(),function())-> ok.
spread_directory(Dir,Top_Directory,ForEachFileFound,ForEachDirFound)
when is_function(ForEachFileFound),is_function(ForEachDirFound) ->
case ?IS_FOLDER(Dir) of
false -> ?NOT_DIR(Dir);
true ->
F = fun(X)->
FileOrDir = filename:absname_join(Dir,X),
case ?IS_FOLDER(FileOrDir) of
true ->
(catch ForEachDirFound(Top_Directory,FileOrDir)),
spawn(fun() ->
?MODULE:spread_directory(FileOrDir,Top_Directory,ForEachFileFound,ForEachDirFound)
end);
false ->
case ?IS_FILE(FileOrDir) of
false -> {error,not_a_file,FileOrDir};
true -> (catch ForEachFileFound(Top_Directory,FileOrDir))
end
end
end,
case file:list_dir(Dir) of {error,_} -> ?FAILED_TO_LIST_DIR(Dir);
{ok,List} -> lists:foreach(F,List)
end
end.
To test it, below is usage:
E:\Applications>erl
Eshell V5.9 (abort with ^G)
1> Dir = "E:/Ruth".
"E:/Ruth"
2> TopDir = "E:/Ruth".
"E:/Ruth"
3> Folder = fun(Parent,F) -> io:format("\n\t~p contains Folder: ~p~n",[Parent,F]) end.
#Fun<erl_eval.12.111823515>
4> File = fun(Parent,F) -> io:format("\n\t~p contains File: ~p~n",[Parent,F]) end.
#Fun<erl_eval.12.111823515>
5> file_scavenger_utilities:spread_directory(Dir,TopDir,File,Folder).
"E:/Ruth" contains File: "e:/Ruth/Thumbs.db"
"E:/Ruth" contains File: "e:/Ruth/Robert Passport.pdf"
"E:/Ruth" contains File: "e:/Ruth/new mixtape.mp3"
"E:/Ruth" contains File: "e:/Ruth/Manning - Java 3d Programming.pdf"
"E:/Ruth" contains File: "e:/Ruth/jcrea350.zip"
"E:/Ruth" contains Folder: "e:/Ruth/java-e books"
"E:/Ruth" contains File: "e:/Ruth/Java Programming Unleashed.pdf"
"E:/Ruth" contains File: "e:/Ruth/Java Programming Language Basics.pdf"
"E:/Ruth" contains File: "e:/Ruth/java-e books/vb blackbook.pdf.pdf"
With my external program I create a PDF file, and try to pass to an erlang program. But the PDF contains EOT symbols. When the first EOT is reached, the transfer stops.
How I can pass the whole file?
added after sarnold,
os:cmd(io_lib:format("LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib;
export LD_LIBRARY_PATH; /home/yola/progra/cpp/pdf/a.out ~s", ["Hello"]))
this doesnt return whole generated file. First EOT symbol encountered at 2305 pos and erlang got 2304 symbols, but file length 64xxx bytes.
The os:cmd function uses EOT to know when to stop reading from the external program; see os.erl. So you'll have to do it in some other way, such as redirecting output to a file:
os:cmd(io_lib:format("LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib;
export LD_LIBRARY_PATH; /home/yola/progra/cpp/pdf/a.out ~s > my.pdf", ["Hello"])),
{ok, MyPDF} = file:read_file("my.pdf")