Where do I place external variables in MQL4 language? - mql4

I am quite new at programming and can't figure out why the code is not compiling and how to make it compile.
It's a simple EA in MetaTrader Terminal 4.
I think the problem is the position, where I declared and initialised variables, I tried placing them under different functions ( indicated with comments in the code ), but still the code wouldn't compile, the code is attached. Any help will be highly appreciated.
#property copyright "Copyright 2017, MetaQuotes Software Corp."
#property link "https://www.mql5.com"
#property version "1.00"
#property strict
extern double ema_red = iMA( NULL, 0, 6, 0, MODE_EMA, PRICE_CLOSE, 0 );
extern double ema_purple = iMA( NULL, 0, 30, 0, MODE_EMA, PRICE_CLOSE, 0 );
extern double ema_blue = iMA( NULL, 0, 8, 1, MODE_EMA, PRICE_CLOSE, 0 );
// int point;
/*
void start()
{
double ema_red = iMA( NULL, 0, 6, 0, MODE_EMA, PRICE_CLOSE, 0 );
double ema_purple = iMA( NULL, 0, 30, 0, MODE_EMA, PRICE_CLOSE, 0 );
double ema_blue = iMA( NULL, 0, 8, 1, MODE_EMA, PRICE_CLOSE, 0 );
return;
}
*/
void OnTick()
{
// double ema_red = iMA( NULL, 0, 6, 0, MODE_EMA, PRICE_CLOSE, 0 );
// double ema_purple = iMA( NULL, 0, 30, 0, MODE_EMA, PRICE_CLOSE, 0 );
// double ema_blue = iMA( NULL, 0, 8, 1, MODE_EMA, PRICE_CLOSE, 0 );
if (OrdersTotal()<=21)
{
if (ema_red == ema_purple)&&(ema_red > ema_blue) // check when the ea's cross/meet
Buy_Order();
if (ema_red == ema_purple)&&(ema_red < ema_blue) // check when the ea's cross/meet
Sell_Order();
}
}
void Buy_Order()
{
double TP = Ask +(PipsToPointFactor()*15);
double SL = (MarketInfo(Symbol(),MODE_TICKVALUE)) - (PipsToPointFactor()*5);
OrderSend(Symbol(),OP_BUY,0.6,Ask,(3*PipsToPointFactor()),SL,TP,"",0,0,Green);
}
void Sell_Order()
{
double TP = (Bid - (15*PipsToPointFactor()));
double SL = ((MarketInfo(Symbol(),MODE_TICKVALUE)) + (PipsToPointFactor()*5 );
OrderSend(Symbol(),OP_SELL,0.6,Bid,(3*PipsToPointFactor()),SL,TP,"",0,0,Red);
}
int OnInit()
{
return(INIT_SUCCEEDED);
}
void OnDeinit(const int reason)
{
}
int PipsToPointFactor() // NOT MY OWN FUNCTION, takes care of pips to points issue
{
int point;
if(Digits==5 || Digits==3) // Check whether it's a 5 digit broker ( 3 digits for Yen )
point=10; // 1 pip to 10 point if 5 digit
else if(Digits==4 || Digits==2)
point=1; // 1 pip to 1 point if 4 digit
return(point);
}

You declare some variables as "extern":
extern double ema_red = iMA(NULL, 0, 6, 0, MODE_EMA, PRICE_CLOSE, 0);
extern double ema_purple = iMA(NULL, 0, 30, 0, MODE_EMA, PRICE_CLOSE, 0);
extern double ema_blue = iMA(NULL, 0, 8, 1, MODE_EMA, PRICE_CLOSE, 0);
You are using them inside functions, inside more than one function, if you count the function start(). So, they need to be defined at least on a file scope. It seems that you define them nowhere.
Define them, in addition to declaring them or just define them:
/* extern */ double ema_red = iMA(NULL, 0, 6, 0, MODE_EMA, PRICE_CLOSE, 0);
/* extern */ double ema_purple = iMA(NULL, 0, 30, 0, MODE_EMA, PRICE_CLOSE, 0);
/* extern */ double ema_blue = iMA(NULL, 0, 8, 1, MODE_EMA, PRICE_CLOSE, 0);
All code files ( those which contain functions which access these variables ) need to "see" the declaration, i.e. the lines with "extern". If you have more than one such code file, that declaration should be in a header.
Exactly one code file then has to do the definition, i.e. the lines without "extern". That is called "global variables".
If you only have one such code file but several others and want to make sure that the variables are not accessible from anywhere else ( a feature for clean coding and for "safety" ), then you do not use a header and restrict the visibility to the one code file doing the definition. You do that by using the keyword "static", exactly where you now have "extern". That is sometimes called "file scope variables".
As another answer said, initialising at definition can only use static values.
If you want to reinitialise at runtime, maybe with usecase specific values, your function "start()" happens to be the way to do it. You just have to change the definition lines to normal write accesses. The function then needs to be called for changing use cases.
Note:
This answer is referring only to C and is older than the edits to the question, which made the MT4-scope more prominent and deleted the C tag.
So be careful if your needs are heavily MT4 related.
Since somebody, including OP, considered the answer useful, I wont delete until a good MT4 answer is available.

This question was originally tagged for C, so that's what this answer addresses.
One immediate issue - objects that are declared at file scope (outside the body of any function), as in
extern double ema_red = iMA(NULL, 0, 6, 0, MODE_EMA, PRICE_CLOSE, 0);
extern double ema_purple = iMA(NULL, 0, 30, 0, MODE_EMA, PRICE_CLOSE, 0);
extern double ema_blue = iMA(NULL, 0, 8, 1, MODE_EMA, PRICE_CLOSE, 0);
may only be initialized with constant expressions (something that can be evaluated at compile time). If iMA is a function call (or a macro that expands into something other than a constant expression), that will cause the compiler to yak.
And, you don't need the extern here.

While syntax may look on a first looksomewhat similar to C,the MQL4 language is by far a different cup of tea,it lives inside a rather specific eco-systemwith many un-paralleled and dual-edge sword features.
TL;DR; at least, you've been warned.
Since the early state of the original MQL4 language, there were important concepts, that go well beyond the classics of C.
The units of MQL4 code ( be it any of { EA: Expert Advisor | CI: Custom Indicator | Script: Script } ) are being executed in several different modes of the code-execution eco-system, either:
having been "put on" a graph, being that during live-market hours or weekends
having been "setup in" a trivial mode in a [Strategy Tester] back-testing facility of the Terminal
having been "setup in" a complex mode of an optimisation module in a [Strategy Tester]
While this may seem obvious for hard-core MetaTrader Quant Modellers, one may also agree this is a hidden part of the O/P context, which was readable just from decoding a bit cryptic second sentence ( cit. orig. ) :
"just a simple EA in MT4".
So, what are the MQL4-ways to use external variables?
All of the { extern | input | sinput } declarations ought be somewhere near the top of the MQL4-code, having a file-level of the scope-of-validity.
Placing the same extern declarations right to the bottom of the file will still work, compiler is aware of the proper scoping ( as it by-design also has to carefully handle all the potential variable name(s) masking(s) / un-masking(s) in case the same names were also used in some call-interface(s) declaration(s) or somewhere "inside" some deeper embedded code-block(s) ).
Such un-orthodox practice should be considered rather in-humane, as we also strive to create a good practice for human-understanding of the code-unit(s) we design, irrespective of compiler's ability to process the tail-definitions the very same way it does for the head-definitions, right?
Besides the trivialities of a variable-definition, belonging to a certain scope-of-validity ( inside a relevant { code-block }-scope ), there are some major implications for extern, input and sinput MQL4-specific declaration modifiers.
//+------------------------------------------------------------------+
//| StackOverflow__test_EA_extern.mq4 |
//| Copyright © 1987-2017 [MS] |
//| nowhere.no |
//+------------------------------------------------------------------+
#property copyright "Copyright © 1987-2017 [MS]"
#property link "nowhere.no"
#property version "1.00"
#property strict // New-MQL4.56789 <--- this CHANGES SOME RUN-TIME FEATURES OF THE SAME SOURCE-CODE (!!!)
//--- extern parameters --- MQL4 declared <extern>-variables have at least DUAL-INTERFACE ROLE
extern double ama_red; // _RED /*| (1) is a LIVE, knowingly bidirectional INTERFACE */
extern double ama_blue; // _BLUE /*| between a code-execution ecosystem */
extern double ama_purple; // _PURPLE /*| & MT4 GUI user-interactions */
// ^ (2) is an ITERATOR, unidirectional INTERFACE ROLE
// ^ from MetaTrader Terminal 4 [StrategyTester]
// ^ into MetaTrader Terminal 4 Optimisation Tool
// ^
// comment^-strings are DISPLAYED in GUI dialogue boxes ( on #property strict // mode )
// + each user-interaction, done via GUI, RESETs the STATE of the so far running EA (!!!)
// + variables are always RE-INITIALISED immediately before the OnInit() is called
// ^^^^^^^^^^^^^^^^^^^^^
//
// - Arrays[] and variables of complex types can't act as <extern>-variables.
/*
//--- input parameters --- New-MQL4.56789 Build 1065+ EA-templates started to be proposed as <input>-based
input double ama_red; // can never be assigned a value to <input>-defined variables
input double ama_blue; // can never be used for a man-machine user-interaction
input double ama_purple; //
*/
#property show_inputs
int aTracePointNUMBER(){ static int aTracePointORDINAL = EMPTY; return( ++aTracePointORDINAL ); }
void aTracePointREPORT( const string aTracePointCALLER,
const double aTracePointDOUBLE_VALUE,
const string aTracePointDOUBLE_NAME
){
PrintFormat( "[%d] In(%s): <_%s_> == %f", // PrintFormat( "[%d] In(%s): <_%s_> == %f",
aTracePointNUMBER(), // aTracePointNUMBER(),
aTracePointCALLER, // __FUNCTION__,
aTracePointDOUBLE_NAME, // "ama_red"
aTracePointDOUBLE_VALUE // ama_red
); // );
}
/*
#include MQL4_Project_common_HEADERS_FILE.mqh // may help with #define-s, but not that much with { extern | input | sinput }-s in MQL4
*/
//+------------------------------------------------------------------+
//| Expert initialization function |
//+------------------------------------------------------------------+
int OnInit() {
aTracePointREPORT( __FUNCTION__, ama_red, "ama_red" );
ama_red = iMA( NULL, 0, 6, 0, MODE_EMA, PRICE_CLOSE, 0 );
aTracePointREPORT( __FUNCTION__, ama_red, "ama_red" );
ama_red = EMPTY;
aTracePointREPORT( __FUNCTION__, ama_red, "ama_red" );
return( INIT_SUCCEEDED ); }
//+------------------------------------------------------------------+
//| Expert deinitialization function |
//+------------------------------------------------------------------+
void OnDeinit( const int reason ) {
// aTracePointREPORT( __FUNCTION__, ama_red, "ama_red" );
}
//+------------------------------------------------------------------+
//| Expert tick function |
//+------------------------------------------------------------------+
void OnTick() {
// uponEntry: // EACH TIME anExternalFxEventSTREAM QUOTE.ARRIVAL
aTracePointREPORT( __FUNCTION__, ama_red, "ama_red" );
// ... EA code ...
ama_red = iMA( NULL, 0, 6, 0, MODE_EMA, PRICE_CLOSE, 0 );
// ...
}
//+------------------------------------------------------------------+
/*
2017.04.15 14:40:45.030 2013.01.01 00:00:00 StackOverflow__test_EA_extern inputs: ama_red=-123456789; ama_blue=-987654321; ama_purple=918273645;
2017.04.15 14:40:45.030 2013.03.21 13:20:00 StackOverflow__test_EA_extern XAUUSD,M1: [0] In(OnInit): <_ama_red_> == -123456789.000000
2017.04.15 14:40:45.030 2013.03.21 13:20:00 StackOverflow__test_EA_extern XAUUSD,M1: [1] In(OnInit): <_ama_red_> == 1608.298571
2017.04.15 14:40:45.030 2013.03.21 13:20:00 StackOverflow__test_EA_extern XAUUSD,M1: [2] In(OnInit): <_ama_red_> == -1.000000
2017.04.15 14:40:45.030 2013.03.21 13:20:00 StackOverflow__test_EA_extern XAUUSD,M1: [3] In(OnTick): <_ama_red_> == -1.000000
2017.04.15 14:43:14.764 2013.03.21 13:20:02 StackOverflow__test_EA_extern XAUUSD,M1: [4] In(OnTick): <_ama_red_> == 1608.298571
2017.04.15 14:43:16.827 2013.03.21 13:20:05 StackOverflow__test_EA_extern XAUUSD,M1: [5] In(OnTick): <_ama_red_> == 1608.296000
2017.04.15 14:43:18.889 2013.03.21 13:20:07 StackOverflow__test_EA_extern XAUUSD,M1: [6] In(OnTick): <_ama_red_> == 1608.293428
2017.04.15 14:43:20.952 2013.03.21 13:20:10 StackOverflow__test_EA_extern XAUUSD,M1: [7] In(OnTick): <_ama_red_> == 1608.295142
2017.04.15 14:43:23.014 2013.03.21 13:20:12 StackOverflow__test_EA_extern XAUUSD,M1: [8] In(OnTick): <_ama_red_> == 1608.296857
2017.04.15 14:43:25.077 2013.03.21 13:20:15 StackOverflow__test_EA_extern XAUUSD,M1: [9] In(OnTick): <_ama_red_> == 1608.293428
*/
By tracing this trivial code, you may realise a few facts:
While one cannot, for obvious compile-time reasons, declare an ema_red et al with being assigned to a non-(compile-time)-constant initialisation, there still is a way to achieve such aimed target and one may use for this a mandatory call to an OnInit(){...} event-handler code-block, to provide such a non-constant value into the intended extern variable.
Listed are the trace-reports from such constructed EA, showing each change of extern-declared inputs, before and after the code-execution reached the event-triggered OnTick(){...} handler ( original start() handler is not used anymore, yet historically still present a lot in MQL4 code-base ):
2017.04.15 14:40:45.030 2013.01.01 00:00:00 StackOverflow__test_EA_extern inputs: ama_red=-123456789; ama_blue=-987654321; ama_purple=918273645;
2017.04.15 14:40:45.030 2013.03.21 13:20:00 StackOverflow__test_EA_extern XAUUSD,M1: [0] In(OnInit): <_ama_red_> == -123456789.000000
2017.04.15 14:40:45.030 2013.03.21 13:20:00 StackOverflow__test_EA_extern XAUUSD,M1: [1] In(OnInit): <_ama_red_> == 1608.298571
2017.04.15 14:40:45.030 2013.03.21 13:20:00 StackOverflow__test_EA_extern XAUUSD,M1: [2] In(OnInit): <_ama_red_> == -1.000000
2017.04.15 14:40:45.030 2013.03.21 13:20:00 StackOverflow__test_EA_extern XAUUSD,M1: [3] In(OnTick): <_ama_red_> == -1.000000
2017.04.15 14:43:14.764 2013.03.21 13:20:02 StackOverflow__test_EA_extern XAUUSD,M1: [4] In(OnTick): <_ama_red_> == 1608.298571
2017.04.15 14:43:16.827 2013.03.21 13:20:05 StackOverflow__test_EA_extern XAUUSD,M1: [5] In(OnTick): <_ama_red_> == 1608.296000
2017.04.15 14:43:18.889 2013.03.21 13:20:07 StackOverflow__test_EA_extern XAUUSD,M1: [6] In(OnTick): <_ama_red_> == 1608.293428
2017.04.15 14:43:20.952 2013.03.21 13:20:10 StackOverflow__test_EA_extern XAUUSD,M1: [7] In(OnTick): <_ama_red_> == 1608.295142
2017.04.15 14:43:23.014 2013.03.21 13:20:12 StackOverflow__test_EA_extern XAUUSD,M1: [8] In(OnTick): <_ama_red_> == 1608.296857
2017.04.15 14:43:25.077 2013.03.21 13:20:15 StackOverflow__test_EA_extern XAUUSD,M1: [9] In(OnTick): <_ama_red_> == 1608.293428
Beware, extern is a BI-DIRECTIONAL INTERFACE (!!!)
Given the above, on condition a user uses the GUI to modify once setup extern to a new value, the code-execution of the currently live-running Expert Advisor is reset to a new initialised values. That may cause a lot of problems in distributed-system's processing and one ought take due care to propagate such "invisible" reset to proper handling of going back to square [ 1 ].
The impact of the live-Market session Expert Advisor code-execution reset could be indeed disastrous, so be always careful, whether your code is being designed as being robust against such unintentional event(s), causing a Deus Ex Machina mystical changes of behaviour, very similar to a sudden thunder from a clear, blue sky.
If one does not expect such habit, a hidden remark in the system journal will typically not suffice for one to realise what happened to the live-session and what cascade of additional side-effects was triggered by such event.
Epilogue:
Upon mastering both the trivial extern and also it's external-iterator alternative for the complex-setup, used in [Strategy Tester] optimiser ( be it a fully-orthogonal brute-force or a somewhat magically labeled Genetic-mode one ), the fully bi-directional interface role of the extern-variables are a very powerful tool for Man-Machine-Interactions during MQL4 Quant Modelling.

Related

Why MQL4 Backtest RSI values do not match with RSI on chart?

I have a simple backtest of below EA:
void OnTick()
{
double rsi = iRSI( Symbol(), PERIOD_M5, 14, PRICE_CLOSE, 0 );
int day = TimeDay( TimeCurrent() );
int hour = TimeHour( TimeCurrent() );
int min = TimeMinute( TimeCurrent() );
if ( day == 7
&& hour >= 9
&& hour < 11
) {
Print( Symbol(), " / ", PERIOD_M5, " rsi: ", (string) rsi );
}
}
However, the backtest log seems does not match with show in the chart as this image:
http://i.imgur.com/PRhtvQD.png
Could you please give some explanation?
Q : "Could you please give some explanation?"
Sure, your code calculates & updates a printed RSI(14)-value (per-tick)
Kindly notice, that the previous bar 08:55 has finished with RSI(14)-value well above the HLINE ~ 30% (if in doubts, one can initially Print( iRSI( Symbol(), PERIOD_M5, 14, PRICE_CLOSE, 1 ) ); where you will see the "previous"-bar value numerically.
From about that value ( above ~ 30% ) the newly opened bar [0] will start to "develop" the actual RSI(14)-value, inside the new bar. So, initially, the values will "move" and the graph plots / redraws the line ( we can visualise each such change as a dot, marker or a Hi/Lo-range ), which is THE REASON, why we finally see a blue line fallen to the position, where the Close[0] has "finished" upon exiting of the bar under review (the 09:00 one, at about the 09:04:59:9999 time).

Why is my EA not moving my position to breakeven?

I'm attempting to modify my market orders to breakeven the position when the position get 100 pips to the good. This also accounts for the StopLevels which are around 20-30 pips for my broker. It checks the param's via a "for(){...} loop" function
The MagicNumber is the timeframe number for the chart it is on (i.e. 240=4H, 60=1H) I don't set a TakeProfit price & initially no StopLoss price.
The EA is not adding a SL to be equal to the opening price when the trade reaches 100 pip in profit (plus stoplevels). Profit points reaches well over 130 points.
My code is below for a OP_SELL order - any help would be appreciated. Regards, Todd
/*Global Declarations*/
double pnlPoints;
double price, sl, tp;
double point;
int stopLevel;
int breakeven;
double newSL;
/*Local Declaratons*/
pnlPoints = 0;
point = MarketInfo( Symbol(), MODE_POINT );
stopLevel = int( MarketInfo( Symbol(), MODE_STOPLEVEL )
+ MarketInfo( Symbol(), MODE_SPREAD )
);
sl = NormalizeDouble( OrderStopLoss(), Digits );
tp = OrderTakeProfit();
breakeven = 100;
for( int s = OrdersTotal() - 1; s >= 0; s-- )
{ if ( ( OrderSelect( s, SELECT_BY_POS, MODE_TRADES ) ) == true )
price = MarketInfo( Symbol(), MODE_ASK );
newSL = NormalizeDouble( OrderOpenPrice(), Digits );
pnlPoints = ( OrderOpenPrice() - price ) / point;
if ( OP_SELL == OrderType() )
if ( Period() == OrderMagicNumber() )
if ( stopLevel < ( newSL - price ) / point )
if ( breakeven < pnlPoints )
if ( newSL != sl )
ModSell = OrderModify( OrderTicket(),
OrderOpenPrice(),
newSL,
tp,
buycolor
);
else if ( ModBuy == false )
{ Print( "OrderModify failed with error #",
GetLastError()
);
}
}
For the moment being,refine the codeandadd self-debuging / tracing code
After OrderModify() use a self-debugging / journaling Print( StringFormat( ... ) ) to document all instructed values used in the actual OrderModify() call and also the remote-execution ( { server-side | StrategyTester } ) reported issues.
The current code does not enter into such self-diagnostics and ModSell is not inspected at all, ModBuy is inspected only at uncertain conditions / by-coincidence at some future visit of the for(){...} code-execution path to a part after newSL == sl ( and all above stated conditions are just by chance met too )
Next, check an assigned value of tp
As stated above,
/*Local Declarations*/
...
tp = OrderTakeProfit();
which introduces a reasonable doubt, that re-using of this ( inherently uncertain value, as no one knows, which OrderSelect() was the last one that set a db.Pool pointer to decide, from which record from the db.Pool this veryOrderTakeProfit() would accidentally read ( if any record is present in db.Pool already ) inside the whole for(){...} traversing the db.Pool records will not meet conditions for setting properly a TakeProfit price in the next series of OrderModify() calls.
This seems to be the root cause, or a source of unhandled exceptions to the valid, Broker-compliant, OrderModify() values.
Try this:
if (newSL != sl ) {
ModSell = OrderModify( OrderTicket(),
OrderOpenPrice(),
OrderOpenPrice(),
0,
OrderExpiration(),
clrRed
);
if(ModBuy == false )
Print( "OrderModify failed with error #", GetLastError());
}
Then check the Expert-tab for error-message if it fails to set the stop.
Also, you need to take note that StopLoss will ONLY occur if you are on the right chart-timeframe; Otherwise, it won't even get into the if-statements.

How to access the second if statement

I am totally drain out as the second if statement couldn't be executed.
My original idea was when volatility is in a range of 90 - 110, the program will send one and only one order. And it will wait and see till the volatility reaches in a range of 111 - 150, and then it will send the second order.
If I don't use a bool function here, the program will send countless order when the range is reached.
Could someone please help me?
if ( TodayMaxVolatilityPercentage >= 90.0
&& ( dayTrend == 1 )
&& orderOpened == false
)
{
Print( "Entered if clause" );
// Print( "Today volatility percentage is: ", TodayMaxVolatilityPercentage + "%" );
// ticket: Returns number of the ticket assigned to the order by the trade server or -1 if it fails.
ticket = OrderSend( Symbol(),
OP_SELL,
0.3,
Bid,
3,
0 * MyPoint,
30 * MyPoint,
NULL,
MagicNumber,
0,
Blue
);
Print( "Order is opened on", OrderOpenTime()+" at price: ", OrderOpenPrice() );
Print( "trend number is ",dayTrend );
if ( ticket > 0 )
{
if ( TakeProfit > 0 ) TheTakeProfit = Bid - TakeProfit * MyPoint;
OrderSelect( ticket, SELECT_BY_TICKET ); // bool value
/* OrderModify( OrderTicket(),
OrderOpenPrice(),
0,
NormalizeDouble( TheTakeProfit, Digits ),
0,
Green
);
*/
}
orderOpened = true;
if ( TodayMaxVolatilityPercentage >= 110.0 ) orderOpened = false;
}
if ( TodayMaxVolatilityPercentage >= 110.0
&& ( dayTrend == 1 )
&& orderOpened == false
)
{
Print( "Entered second if clause" );
// ticket: Returns number of the ticket assigned to the order by the trade server or -1 if it fails.
ticket = OrderSend( Symbol(),
OP_SELL,
0.3,
Bid,
3,
0 * MyPoint,
30 * MyPoint,
NULL,
MagicNumber,
0,
Blue
);
if ( ticket > 0 )
{
if ( TakeProfit > 0 ) TheTakeProfit = Bid - TakeProfit * MyPoint;
OrderSelect( ticket, SELECT_BY_TICKET ); // bool value
/* OrderModify( OrderTicket(),
OrderOpenPrice(),
0,
NormalizeDouble( TheTakeProfit, Digits ),
0,
Green
);
*/
}
orderOpened = true;
}
A hidden show-stopper:
By design, the first successful OrderSend() returns a value you assign into an ticket = OrderSend(...).
The devil is hidden in detail.
[MQL4-Help] says:
Returns number of the ticket assigned to the order by the trade server or -1 if it fails.
So the first successful OrderSend() ticket# returned is 0 ( in StrategyTester, a real MT4/Server rather runs into high numbers drawn from it's db.pool of tickets ).
Thus the second if ( ticket > 0 ) will never let you in.
It is common to rather use if ( ticket > EMPTY ) so as to also clarify the intention and use symbolic instead of a -1 integer constant.

retrieve the grammar rules from the generated parsing tables

I have a quite old C corporate parser code that was generated from an ancient Yacc and uses the yyact, yypact, yypgo, yyr1, yyr2, yytoks, yyexca, yychk, yydef tables (but no yyreds though) and the original grammar source is lost. That legacy piece of code need revamping but I cannot afford to recode it from scratch.
Could it be possible to mechanically retrieve / regenerate the parsing rules by deduction of the parsing tables in order to reconstruct the grammar?
Example with a little expression parser that I can process with the same ancient Yacc:
yytabelem yyexca[] ={
-1, 1,
0, -1,
-2, 0,
-1, 21,
261, 0,
-2, 8,
};
yytabelem yyact[]={
13, 9, 10, 11, 12, 23, 8, 22, 13, 9,
10, 11, 12, 9, 10, 11, 12, 1, 2, 11,
12, 6, 7, 4, 3, 0, 16, 5, 0, 14,
15, 0, 0, 0, 17, 18, 19, 20, 21, 0,
0, 24 };
yytabelem yypact[]={
-248, -1000, -236, -261, -236, -236, -1000, -1000, -248, -236,
-236, -236, -236, -236, -253, -1000, -263, -245, -245, -1000,
-1000, -249, -1000, -248, -1000 };
yytabelem yypgo[]={
0, 17, 24 };
yytabelem yyr1[]={
0, 1, 1, 1, 2, 2, 2, 2, 2, 2,
2, 2, 2 };
yytabelem yyr2[]={
0, 8, 12, 0, 6, 6, 6, 6, 6, 6,
4, 2, 2 };
yytabelem yychk[]={
-1000, -1, 266, -2, 259, 263, 257, 258, 267, 262,
263, 264, 265, 261, -2, -2, -1, -2, -2, -2,
-2, -2, 260, 268, -1 };
yytabelem yydef[]={
3, -2, 0, 0, 0, 0, 11, 12, 3, 0,
0, 0, 0, 0, 0, 10, 1, 4, 5, 6,
7, -2, 9, 3, 2 };
yytoktype yytoks[] =
{
"NAME", 257,
"NUMBER", 258,
"LPAREN", 259,
"RPAREN", 260,
"EQUAL", 261,
"PLUS", 262,
"MINUS", 263,
"TIMES", 264,
"DIVIDE", 265,
"IF", 266,
"THEN", 267,
"ELSE", 268,
"LOW", 269,
"UMINUS", 270,
"-unknown-", -1 /* ends search */
};
/* am getting this table in my example,
but it is not present in the studied parser :^( */
char * yyreds[] =
{
"-no such reduction-",
"stmt : IF exp THEN stmt",
"stmt : IF exp THEN stmt ELSE stmt",
"stmt : /* empty */",
"exp : exp PLUS exp",
"exp : exp MINUS exp",
"exp : exp TIMES exp",
"exp : exp DIVIDE exp",
"exp : exp EQUAL exp",
"exp : LPAREN exp RPAREN",
"exp : MINUS exp",
"exp : NAME",
"exp : NUMBER",
};
I am looking to retrieve
stmt : IF exp THEN stmt
| IF exp THEN stmt ELSE stmt
| /*others*/
;
exp : exp PLUS exp
| exp MINUS exp
| exp TIMES exp
| exp DIVIDE exp
| exp EQUAL exp
| LPAREN exp RPAREN
| MINUS exp
| NAME
| NUMBER
;
Edit: I have stripped down the generated parser of my example for clarity, but to help some analysis i have published the whole generated code as a gist. Please not that for some unknown reason there is no yyreds table in the parser I am trying to study / change. I suppose it would not have been fun :^S
An interesting problem. Just from matching the tables to the grammar, it seems that yyr1 and yyr2 give you the "outline" of the rules -- yyr1 is the symbol on the left side of each rule, while yyr2 is 2x the number of symbols on the right side. You also have the names of all the terminals in a convenient table. But the names of the non-terminals are lost.
To figure out which symbols go on the rhs of each rule, you'll need to reconstruct the state machine from the tables, which likely involves reading and understanding the code in the y.tab.c file that actually does the parsing. Some of the tables (looks like yypact, yychk and yydef) are indexed by state number. It seems likely that yyact is indexed by yypact[state] + token. But those are only guesses. You need to look at the parsing code and understand how its using the tables to encode possible shifts, reduces, and gotos.
Once you have the state machine, you can backtrack from the states containing reductions of specific rules through the states that have shifts and gotos of that rule. A shift into a reduction state means the last symbol on the rhs of that rule is the token shifted. A goto into a reduction state means the last symbol on the rhs is symbol for the goto. The second-to-last symbol comes from the shift/goto to the state that does the shift/goto to the reduction state, and so on.
edit
As I surmised, yypact is the 'primary action' for a state. If the value is YYFLAG (-1000), this is a reduce-only state (no shifts). Otherwise it is a potential shift state and yyact[yypact[state] + token] gives you the potential state to shift to. If yypact[state] + token is out of range for the yyact table, or the token doesn't match the entry symbol for that state, then there's no shift on that token.
yychk is the entry symbol for each state -- a positive number means you shift to that state on that token, while a negative means you goto that state on that non-terminal.
yydef is the reduction for that state -- a positive number means reduce that rule, while 0 means no reduction, and -2 means two or more possible reductions. yyexca is the table of reductions for those states with more than one reduction. The pair -1 state means the following entries are for the given state; following pairs of token rule mean that for lookahead token it should reduce rule. A -2 for token is a wildcard (end of the list), while a 0 for the rule means no rule to reduce (an error instead), and -1 means accept the input.
The yypgo table is the gotos for a symbol -- you go to state yyact[yypgo[sym] + state + 1] if that's in range for yyact and yyact[yypgo[sym]] otherwise.
So to reconstruct rules, look at the yydef and yyexca tables to see which states reduce each rule, and go backwards to see how the state is reached.
For example, rule #1. From the yyr1 and yyr2 tables, we know its of the form S1: X X X X -- non-terminal #1 on the lhs and 4 symbols on the rhs. Its reduced in state 16 (from the yydef table, and the accessing symbol for state 16 (from yychk) is -1. So its:
S1: ?? ?? ?? S1
You get into state 16 from yyact[26], and yypgo[1] == 17, so that means the goto is coming from state 8 (26 == yypgo[1] + 8 + 1. The accessing symbol of state 8 is 267 (THEN) so now we have:
S1: ?? ?? THEN S1
You get into state 8 from yyact[6], so the previous state has yypact[state] == -261 which is state 3. yychk[3] == -2, so we have:
S1: ?? S2 THEN S1
You get into state 3 from yyact[24], and yypgo[2] == 24 so any state might goto 3 here. So we're now kind of stuck for this rule; to figure out what the first symbol is, we need to work our way forward from state 0 (the start state) to reconstruct the state machine.
edit
This code will decode the state machine from the table format above and print out all the shift/reduce/goto actions in each state:
#define ALEN(A) (sizeof(A)/sizeof(A[0]))
for (int state = 0; state < ALEN(yypact); state++) {
printf("state %d:\n", state);
for (int i = 0; i < ALEN(yyact); i++) {
int sym = yychk[yyact[i]];
if (sym > 0 && i == yypact[state] + sym)
printf("\ttoken %d shift state %d\n", sym, yyact[i]);
if (sym < 0 && -sym < ALEN(yypgo) &&
(i == yypgo[-sym] || i == yypgo[-sym] + state + 1))
printf("\tsymbol %d goto state %d\n", -sym, yyact[i]); }
if (yydef[state] > 0)
printf("\tdefault reduce rule %d\n", yydef[state]);
if (yydef[state] < 0) {
for (int i = 0; i < ALEN(yyexca); i+= 2) {
if (yyexca[i] == -1 && yyexca[i+1] == state) {
for (int j = i+2; j < ALEN(yyexca) && yyexca[j] != -1; j += 2) {
if (yyexca[j] < 0) printf ("\tdefault ");
else printf("\ttoken %d ", yyexca[j]);
if (yyexca[j+1] < 0) printf("accept\n");
else if(yyexca[j+1] == 0) printf("error\n");
else printf("reduce rule %d\n", yyexca[j+1]); } } } } }
It will produce output like:
state 0:
symbol 1 goto state 1
token 266 shift state 2
symbol 2 goto state 3
default reduce rule 3
state 1:
symbol 1 goto state 1
symbol 2 goto state 3
token 0 accept
default error
state 2:
symbol 1 goto state 1
token 257 shift state 6
token 258 shift state 7
token 259 shift state 4
symbol 2 goto state 3
token 263 shift state 5
state 3:
token 261 shift state 13
token 262 shift state 9
token 263 shift state 10
token 264 shift state 11
token 265 shift state 12
token 267 shift state 8
symbol 1 goto state 1
symbol 2 goto state 3
..etc
which should be helpful for reconstructing the grammar.

How can I access my constant memory in my kernel?

I can't manage to access the data in my constant memory and I don't know why. Here is a snippet of my code:
#define N 10
__constant__ int constBuf_d[N];
__global__ void foo( int *results, int *constBuf )
{
int tdx = threadIdx.x;
int idx = blockIdx.x * blockDim.x + tdx;
if( idx < N )
{
results[idx] = constBuf[idx];
}
}
// main routine that executes on the host
int main(int argc, char* argv[])
{
int *results_h = new int[N];
int *results_d = NULL;
cudaMalloc((void **)&results_d, N*sizeof(int));
int arr[10] = { 16, 2, 77, 40, 12, 3, 5, 3, 6, 6 };
int *cpnt;
cudaError_t err = cudaGetSymbolAddress((void **)&cpnt, "constBuf_d");
if( err )
cout << "error!";
cudaMemcpyToSymbol((void**)&cpnt, arr, N*sizeof(int), 0, cudaMemcpyHostToDevice);
foo <<< 1, 256 >>> ( results_d, cpnt );
cudaMemcpy(results_h, results_d, N*sizeof(int), cudaMemcpyDeviceToHost);
for( int i=0; i < N; ++i )
printf("%i ", results_h[i] );
}
For some reason, I only get "0" in results_h. I'm running CUDA 4.0 with a card with capability 1.1.
Any ideas? Thanks!
If you add proper error checking to your code, you will find that the cudaMemcpyToSymbol is failing with a invalid device symbol error. You either need to pass the symbol by name, or use cudaMemcpy instead. So this:
cudaGetSymbolAddress((void **)&cpnt, "constBuf_d");
cudaMemcpy(cpnt, arr, N*sizeof(int), cudaMemcpyHostToDevice);
or
cudaMemcpyToSymbol("constBuf_d", arr, N*sizeof(int), 0, cudaMemcpyHostToDevice);
or
cudaMemcpyToSymbol(constBuf_d, arr, N*sizeof(int), 0, cudaMemcpyHostToDevice);
will work. Having said that, passing a constant memory address as an argument to a kernel is the wrong way to use constant memory - it defeats the compiler from generating instructions to access memory via the constant memory cache. Compare the compute capability 1.2 PTX generated for your kernel:
.entry _Z3fooPiS_ (
.param .u32 __cudaparm__Z3fooPiS__results,
.param .u32 __cudaparm__Z3fooPiS__constBuf)
{
.reg .u16 %rh<4>;
.reg .u32 %r<12>;
.reg .pred %p<3>;
.loc 16 7 0
$LDWbegin__Z3fooPiS_:
mov.u16 %rh1, %ctaid.x;
mov.u16 %rh2, %ntid.x;
mul.wide.u16 %r1, %rh1, %rh2;
cvt.s32.u16 %r2, %tid.x;
add.u32 %r3, %r2, %r1;
mov.u32 %r4, 9;
setp.gt.s32 %p1, %r3, %r4;
#%p1 bra $Lt_0_1026;
.loc 16 14 0
mul.lo.u32 %r5, %r3, 4;
ld.param.u32 %r6, [__cudaparm__Z3fooPiS__constBuf];
add.u32 %r7, %r6, %r5;
ld.global.s32 %r8, [%r7+0];
ld.param.u32 %r9, [__cudaparm__Z3fooPiS__results];
add.u32 %r10, %r9, %r5;
st.global.s32 [%r10+0], %r8;
$Lt_0_1026:
.loc 16 16 0
exit;
$LDWend__Z3fooPiS_:
} // _Z3fooPiS_
with this kernel:
__global__ void foo2( int *results )
{
int tdx = threadIdx.x;
int idx = blockIdx.x * blockDim.x + tdx;
if( idx < N )
{
results[idx] = constBuf_d[idx];
}
}
which produces
.entry _Z4foo2Pi (
.param .u32 __cudaparm__Z4foo2Pi_results)
{
.reg .u16 %rh<4>;
.reg .u32 %r<12>;
.reg .pred %p<3>;
.loc 16 18 0
$LDWbegin__Z4foo2Pi:
mov.u16 %rh1, %ctaid.x;
mov.u16 %rh2, %ntid.x;
mul.wide.u16 %r1, %rh1, %rh2;
cvt.s32.u16 %r2, %tid.x;
add.u32 %r3, %r2, %r1;
mov.u32 %r4, 9;
setp.gt.s32 %p1, %r3, %r4;
#%p1 bra $Lt_1_1026;
.loc 16 25 0
mul.lo.u32 %r5, %r3, 4;
mov.u32 %r6, constBuf_d;
add.u32 %r7, %r5, %r6;
ld.const.s32 %r8, [%r7+0];
ld.param.u32 %r9, [__cudaparm__Z4foo2Pi_results];
add.u32 %r10, %r9, %r5;
st.global.s32 [%r10+0], %r8;
$Lt_1_1026:
.loc 16 27 0
exit;
$LDWend__Z4foo2Pi:
} // _Z4foo2Pi
Note that in the second case, constBuf_d is accessed via ld.const.s32, rather than ld.global.s32, so that constant memory cache is used.
Excellent answer #talonmies. But I would like to mention that there have been changes in cuda 5. In the function MemcpyToSymbol(), char * argument is no longer supported.
The CUDA 5 release notes read:
** The use of a character string to indicate a device symbol, which was possible with certain API functions, is no longer supported. Instead, the symbol should be used directly.
Instead the copy have to be made to the constant memory as follows :
cudaMemcpyToSymbol( dev_x, x, N * sizeof(float) );
In this case "dev_x" is pointer to constant memory and "x" is pointer to host memory which needs to be copied into dev_x.

Resources