After implementing a type checker in Rascal using TypePal, how do I hook it up to the Eclipse IDE? Any resources or repositories to the solution to this problem would be appreciated, Thanks.
in Eclipse search path of the Rascal Navigator view in your project you will find the rascal_eclipse library which contains a good example: demo::lang::Pico::Plugin
in this module you see how to register a language with Eclipse:
registerLanguage("Pico Language", "pico", parsePico); where parsePico is a function reference. Pass your own parameters here. The registerLanguage function comes from util::IDE.
now you can open files with the "IMP editor" in Eclipse and they will use your parser and show syntax highlighting.
next up is registering other effects with the IDE. The library function to call is registerAnnotator. You pass it a function that takes a parse tree for your language and annotates it with error messages:
the messages may be distributed over the tree, using #message or
using a list of #messages at the top of the tree
the error messages will be added as annotations in the editor and registered with the Problem View automatically.
So you have to wire the output of TypePal into the annotator function yourself. Which should be a one-liner.
Alternatively, running type-checks can also be useful after a "save" action. In this case you can register another type of contribition (also in the Pico demo), called builder: builder(set[Message] ((&T<:Tree) tree) messages), and register that with the registerContributions function.
The Message ADT is the same for the annotator and the builder. Both have the effect of adding editor annotations and problems in the view.
Here is some example code taken from an older open-source DSL project called "Bird", https://github.com/SWAT-engineering/bird:
Tree checkBird(Tree input){
model = birdTModelFromTree(input, pathConf = config(input#\loc)); // your function that collects & solves
types = getFacts(model);
return input[#messages={*getMessages(model)}]
[#hyperlinks=getUseDef(model)]
[#docs=(l:"<prettyPrintAType(types[l])>" | l <- types)]
;
}
birdTModelFromTree calls collectAndSolve from TypePal and returns the TModel
getMessage comes from TypePal's Utilities and extracts a list[Message] from the TModel that can be directly communicated to Eclipse via the #messages annotation.
The entire checkBird function is registered as an annotator using the util::IDE function registerAnnotator.
This code hasn't been used for a while, so if you run into trouble, please contact again?
In pyspark reading csv files from different paths gets failed if even one path does not exist.
Logs = spark.read.load(Logpaths, format="csv", schema=logsSchema, header="true", mode="DROPMALFORMED");
Here Logpaths is an array that contain multiple paths. And these paths are created dynamically depending upon given startDate and endDate range. If Logpaths contain 5 paths and first 3 exists but 4th does not exist. Then whole extraction gets failed. How can I avoid this in pyspark or how can I check there existance before reading?
In scala I did this by checking file existance and filter out non-existed records by using hadoop hdfs filesystem globStatus function.
Path = '/bilal/2018.12.16/logs.csv'
val hadoopConf = new org.apache.hadoop.conf.Configuration()
val fs = org.apache.hadoop.fs.FileSystem.get(hadoopConf)
val fileStatus = fs.globStatus(new org.apache.hadoop.fs.Path(Path));
So I got what I was looking for. Like the code I posted in the question which can be used in scala for file existance check. We can use below code in case of PySpark.
fs = sc._jvm.org.apache.hadoop.fs.FileSystem.get(sc._jsc.hadoopConfiguration())
fs.exists(sc._jvm.org.apache.hadoop.fs.Path("bilal/logs/log.csv"))
This is exactly the same code also used in scala, so in this case we are using java library for hadoop and java code runs on JVM on which spark is running.
I have setup Jenkins to run pylint on all python source files and all the log files are generated (apparently correctly) into a sub-directory as follows:
Source\pylint_logs\pylint1.log, pylint2.log, ..., pylint75.log
I have included a --msg-template definition based on the instructions on my Jenkins Configure page: Post-build Actions->Record compiler warnings and static analysis results->Static Analysis Tools. The template is shown as:
msg-template={path}:{line}: [{msg_id}, {obj}] {msg} ({symbol})
An example of one of the log files being generated by Jenkins/pylint is as follows:
************* Module FigureView
myapp\Views\FigureView.py:1: [C0103, ] Module name "FigureView" doesn't conform to snake_case naming style (invalid-name)
myapp\Views\FigureView.py:30: [C0103, FigureView.__init__] Attribute name "ax" doesn't conform to snake_case naming style (invalid-name)
------------------------------------------------------------------
Your code has been rated at 8.57/10 (previous run: 8.57/10, +0.00)
For the PyLint Report File Pattern, I have: Source/pylint_logs/pylint*.log
It appears that PyLint Warnings is parsing the files because the console output looks like this:
[PyLint] Searching for all files in 'D:\Jenkins\workspace\PROJECT' that match the pattern 'Source/pylint_logs/pylint*.log'
[PyLint] -> found 75 files
[PyLint] Successfully parsed file D:\Jenkins\workspace\PROJECT\Source\pylint_logs\pylint1.log
[PyLint] -> found 0 issues (skipped 0 duplicates)
[PyLint] Successfully parsed file D:\Jenkins\workspace\PROJECT\Source\pylint_logs\pylint10.log
[PyLint] -> found 0 issues (skipped 0 duplicates)
This repeats for all 75 files, even though there are plenty of issues in the log files.
What is odd, is that when I was first prototyping the use of Jenkins on this project, I set it up to just run pylint on a single file. I ran across another StackOverflow post that showed a msg-template that allowed me to get it working (unable to get pylint output to populate the violations graph). I even got the graph to show up for the PyLint Warnings Trend. I used the following definition per the post:
msg-template={path}:{line}: [{msg_id}({symbol}), {obj}] {msg}
Note that this format is slightly different from the one recommended by my Jenkins page (shown earlier). Even though this worked for a single file, neither template now seems to work for multiple files, or else there is something other than the template causing the problem. My graph has flat-lined, and I always get 0 issues reported.
I have had trouble finding useful documentation on the Jenkins PyLint Warnings tool. Does anyone have any ideas or pointers to documentation I can research further? Thanks much!
Ensure pass output-format parameter in pylint command. Example:
pylint --exit-zero --output-format=parseable module1 module2 > pylint.report
you have to set the Pylint's option --message-template in .pylintrc as
msg-template={path}: {line}: [{msg_id} ({symbol}), {obj}] {msg}
output-format=text
I am new to message broker development. I tried to convert source SOAP over xml file to target SOAP over xml file.On my message flow source message discarded to catch terminal.I am not able to find out the problem
my flow : MQINPUT NODE ---> COMPUTE NODE --> MQOUTPUT NODE
If any provide solution on this that may me helpful for me.
DECLARE soapenv CHARACTER 'SOAP-ENV';
SET OutputRoot.XMNLSC.soapenv:Envelope.soapenv:Body.params.ORIGIN_TYPE_CD = InputRoot.XMNLSC.soapenv:Envelope.soapenv:Body.params.originType;
**
Your first line is definitely wrong, but you should be able to see that from the exceptions you are getting.
The first line should be:
DECLARE soapenv NAMESPACE 'http://schemas.xmlsoap.org/soap/envelope/';
An in the further lines the domain should be XMLNSC not XMNLSC.
I was wondering if you know of any way to use prolog for stream processing, that is, some kind of reactive programming, or at least to let a query run on a knowledge base that is continuously updated (effectively a stream), and continuously output the output of the reasoning?
Anything implemented in the popular "prologs", such as SWI-prolog?
You can use Logtalk's support for event-driven programming to define monitors that watch for knowledge base update events and react accordingly. You can run Logtalk using most Prolog systems as the backed compiler, including SWI-Prolog.
The event-driven features are described e.g. in the user manual:
http://logtalk.org/manuals/userman/events.html
The current distribution contains some examples of using events and monitors. An interesting one considering your question is the bricks example:
https://github.com/LogtalkDotOrg/logtalk3/tree/master/examples/bricks
Running this example first and then looking at its code should give you as good idea of what you can do with system wide events and monitors.
XSB has stream processing capabilities. See page 14 in the
XSB Manual
I'm working on something related, in project pqConsole already there is the basic capability: report to the user structured data, containing actionable areas (links) that call back in Prolog current state, hence the possibility to expose actions and react appropriately (hopefully).
It's strictly related to pqConsole::win_write_html, showcasing recent Qt capabilities of SWI-Prolog.
Here an example of a snippet producing only a simple formatted report, I'll try now to add the reactive part, so you can evaluate if you find expressive this basic system. Hints are welcome...
/* File: win_html_write_test.pl
Author: Carlo,,,
Created: Aug 27 2013
Purpose: example usage win_html_write/1
*/
:- module(win_html_write_test,
[dir2list/1
]).
:- [library(http/html_write)].
:- [library(dirtree)].
dir2list(Path) :-
dirtree(Path, DirTree),
% sortree(compare_by_attr(name), DirTree, Sorted), !,
DirTree = Sorted,
phrase(html([\css,
\logo,
hr([]),
ul(\dirtree2html(Sorted, [])),
br([])]), Tokens),
with_output_to(atom(X), print_html(Tokens)),
win_html_write(X),
dump_page_to_debug(X).
css --> html(style(type='text/css',
['.size{color:blue;}'
])).
logo --> html(img([src=':/swipl.png'],[])).
dirtree2html(element(dir, A, S), Parents) -->
html(li([\elem2html(A),
ul(\elements2html(S, [A|Parents]))])).
dirtree2html(element(file, A, []), _Parents) -->
html(li(\elem2html(A))).
elem2html(A) -->
{memberchk(name=N, A),
memberchk(size=S, A)
},
html([span([class=name], N), ' : ', span([class=size], S)]).
elements2html([E|Es], Parents) -->
dirtree2html(E, Parents),
elements2html(Es, Parents).
elements2html([], _Parents) --> [].
dump_page_to_debug(X) :-
open('page_to_debug.html', write, S),
format(S, '<html>~n~s~n</html>~n', [X]),
close(S).
This snippet depends on dirtree, that should be installed with
?- pack_install(dirtree).
edit With 3 modifications now the report has the ability to invoke editing of files:
call to get paths in structure
dir2list(Path) :-
dirtree(Path, DirTreeT),
assign_path(DirTreeT, DirTree),
...
request a specialized output for files only
dirtree2html(element(file, A, []), _Parents) -->
html(li(\file2html(A))).
finally, the 'handler' - here just place a request to invoke the editor
file2html(A) -->
{memberchk(name=N, A),
memberchk(path=P, A),
memberchk(size=S, A)
},
html([span([class=name],
[a([href='writeln(editing(\'~s\')), edit(\'~s\')'-[N,P]], N)]
), ' : ', span([class=size], S)]).
and now the file names are clickable, write a message and get edited if requested: a picture
You should check out RTEC: Run-Time Event Calculus:
https://github.com/aartikis/RTEC
RTEC is an open-source Event Calculus dialect optimised for stream reasoning. It is written in Prolog and has been tested under YAP 6.2.
Feature highlights:
Interval-based.
Sliding window reasoning.
Interval manipulation constructs for non-inertial fluents.
Caching for hierarchical knowledge bases.
Support for out-of-order data streams.
Indexing for handling efficiently irrelevant data.
There is also a mention of it on the SWI-Prolog website:
https://www.swi-prolog.org/pack/file_details/prologmud_I7/prolog/ec_planner/RTEC/README.md
which presumably relies on:
https://www.swi-prolog.org/pldoc/doc/_SWI_/library/dialect/yap.pl
I don't know why this hasn't been brought up so far, but in SWI-Prolog there is prolog_listen, which can, amongst other things, monitor dynamic updates to the database: