ARM64 - GNU Assembler: Padding with NOP's to next label - arm64

Sorry, I don't know how to correctly formulate this question. But I will try it with a small example.
I'm searching for a directive in GAS to add a specific amount of padding instructions (NOP's) to the next label. The following example is a extract of the vector table for the AArch64 architecture. Each entry for the handler should be 32 instructions (128 bytes) long.
My goal:
_vec_tbl_cur_sp0:
_vec_tbl_cur_sp0_syn:
mov x0, #0x0
bl _vec_handler_syn
eret
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
_vec_tbl_cur_sp0_irq:
Here I want, that GAS fill the table with the correct amount of NOP's after the ERET instruction to the next label _vec_tbl_cur_sp0_irq.
Is there a directive or macro in GAS to do this?

I have found the solution. The required directive is .balgin.
.balign[wl] abs-expr, abs-expr, abs-expr
Pad the location counter (in the current subsection) to a particular
storage boundary. The first expression (which must be absolute) is the
alignment request in bytes. For example `.balign 8' advances the
location counter until it is a multiple of 8. If the location counter
is already a multiple of 8, no change is needed.
The second expression (also absolute) gives the fill value to be
stored in the padding bytes. It (and the comma) may be omitted. If it
is omitted, the padding bytes are normally zero. However, on some
systems, if the section is marked as containing code and the fill
value is omitted, the space is filled with no-op instructions.
The third expression is also absolute, and is also optional. If it is
present, it is the maximum number of bytes that should be skipped by
this alignment directive. If doing the alignment would require
skipping more bytes than the specified maximum, then the alignment is
not done at all. You can omit the fill value (the second argument)
entirely by simply using two commas after the required alignment; this
can be useful if you want the alignment to be filled with no-op
instructions when appropriate.
The .balignw and .balignl directives are variants of the .balign
directive. The .balignw directive treats the fill pattern as a two
byte word value. The .balignl directives treats the fill pattern as a
four byte longword value. For example, .balignw 4,0x368d will align to
a multiple of 4. If it skips two bytes, they will be filled in with
the value 0x368d (the exact placement of the bytes depends upon the
endianness of the processor). If it skips 1 or 3 bytes, the fill value
is undefined.
Quote from the GAS manual
The following works as excepted:
_vec_tbl_s:
_vec_tbl_cur_sp0:
_vec_tbl_cur_sp0_syn:
mov x0, #0x0
bl _vec_handler_syn
eret
.balign 128
_vec_tbl_cur_sp0_irq:
mov x0, #0x0
bl _vec_handler_irq
eret
.balign 128
_vec_tbl_cur_sp0_fiq:
mov x0, #0x0
bl _vec_handler_fiq
eret
.balign 128
_vec_tbl_cur_sp0_err:
mov x0, #0x0
bl _vec_handler_err
eret
.balign 128
The disassembled binary:
$ aarch64-none-elf-objdump -d vector.o
vector.o: file format elf64-littleaarch64
Disassembly of section .text:
0000000000000000 <_vec_tbl_cur_sp0>:
0: d2800000 mov x0, #0x0 // #0
4: 940001ff bl 800 <_vec_handler_syn>
8: d69f03e0 eret
c: d503201f nop
10: d503201f nop
14: d503201f nop
18: d503201f nop
1c: d503201f nop
20: d503201f nop
24: d503201f nop
28: d503201f nop
2c: d503201f nop
30: d503201f nop
34: d503201f nop
38: d503201f nop
3c: d503201f nop
40: d503201f nop
44: d503201f nop
48: d503201f nop
4c: d503201f nop
50: d503201f nop
54: d503201f nop
58: d503201f nop
5c: d503201f nop
60: d503201f nop
64: d503201f nop
68: d503201f nop
6c: d503201f nop
70: d503201f nop
74: d503201f nop
78: d503201f nop
7c: d503201f nop
0000000000000080 <_vec_tbl_cur_sp0_irq>:
80: d2800000 mov x0, #0x0 // #0
84: 940001e1 bl 808 <_vec_handler_irq>
88: d69f03e0 eret
8c: d503201f nop
90: d503201f nop
94: d503201f nop
98: d503201f nop
9c: d503201f nop
a0: d503201f nop
a4: d503201f nop
a8: d503201f nop
ac: d503201f nop
b0: d503201f nop
b4: d503201f nop
b8: d503201f nop
bc: d503201f nop
c0: d503201f nop
c4: d503201f nop
c8: d503201f nop
cc: d503201f nop
d0: d503201f nop
d4: d503201f nop
d8: d503201f nop
dc: d503201f nop
e0: d503201f nop
e4: d503201f nop
e8: d503201f nop
ec: d503201f nop
f0: d503201f nop
f4: d503201f nop
f8: d503201f nop
fc: d503201f nop
0000000000000100 <_vec_tbl_cur_sp0_fiq>:
100: d2800000 mov x0, #0x0 // #0
104: 940001c3 bl 810 <_vec_handler_fiq>
108: d69f03e0 eret
10c: d503201f nop
110: d503201f nop
114: d503201f nop
118: d503201f nop
11c: d503201f nop
120: d503201f nop
124: d503201f nop
128: d503201f nop
12c: d503201f nop
130: d503201f nop
134: d503201f nop
138: d503201f nop
13c: d503201f nop
140: d503201f nop
144: d503201f nop
148: d503201f nop
14c: d503201f nop
150: d503201f nop
154: d503201f nop
158: d503201f nop
15c: d503201f nop
160: d503201f nop
164: d503201f nop
168: d503201f nop
16c: d503201f nop
170: d503201f nop
174: d503201f nop
178: d503201f nop
17c: d503201f nop
0000000000000180 <_vec_tbl_cur_sp0_err>:
180: d2800000 mov x0, #0x0 // #0
184: 940001a5 bl 818 <_vec_handler_err>
188: d69f03e0 eret
18c: d503201f nop
190: d503201f nop
194: d503201f nop
198: d503201f nop
19c: d503201f nop
1a0: d503201f nop
1a4: d503201f nop
1a8: d503201f nop
1ac: d503201f nop
1b0: d503201f nop
1b4: d503201f nop
1b8: d503201f nop
1bc: d503201f nop
1c0: d503201f nop
1c4: d503201f nop
1c8: d503201f nop
1cc: d503201f nop
1d0: d503201f nop
1d4: d503201f nop
1d8: d503201f nop
1dc: d503201f nop
1e0: d503201f nop
1e4: d503201f nop
1e8: d503201f nop
1ec: d503201f nop
1f0: d503201f nop
1f4: d503201f nop
1f8: d503201f nop
1fc: d503201f nop
Sorry, asked too quickly.

Related

CSV::MalformedCSVError: New line must be <"\n\r">

Trying to parse this file with Ruby CSV.
https://www.sec.gov/files/data/broker-dealers/company-information-about-active-broker-dealers/bd070219.txt
However, I am getting an error.
CSV.open(file_name, "r", { :col_sep => "\t", :row_sep => "\n\r" }).each do |row|
puts row
end
CSV::MalformedCSVError: New line must be <"\n\r"> not <"\r"> in line
1.
Windows row_sep is "\r\n", not "\n\r". However this CSV is malformed. Looking at it using a hex editor it appears to be using "\r\r\n".
It's tab-delimited.
In addition it is not using proper quoting, line 247 has 600 "B" STREET STE. 2204, so you need to turn off quote characters.
quote_char: nil, col_sep: "\t", row_sep: "\r\r\n"
There's an extra tab on the end, each line ends with \t\r\r\n. You can also look at it as using a row_sep of "\r\n" with an extra \r field.
quote_char: nil, col_sep: "\t", row_sep: "\r\n"
Or you can view it as having a row_sep of \t\r\r\n and no extra field.
quote_char: nil, col_sep: "\t", row_sep: "\t\r\r\n"
Either way, it's a mess.
I used a hex editor to look at the file as text and raw data side by side. This let me see what's truly at the end of the line.
87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
00000000: 3030 3030 3030 3139 3034 0941 4252 4148 0000001904.ABRAH
00000010: 414d 2053 4543 5552 4954 4945 5320 434f AM SECURITIES CO
00000020: 5250 4f52 4154 494f 4e09 3030 3832 3934 RPORATION.008294
00000030: 3532 0933 3732 3420 3437 5448 2053 5452 52.3724 47TH STR
00000040: 4545 5420 4354 2e20 4e57 0920 0947 4947 EET CT. NW. .GIG
00000050: 2048 4152 424f 5209 5741 0939 3833 3335 HARBOR.WA.98335
00000060: 090d 0d0a 3030 3030 3030 3233 3033 0950 ....0000002303.P
^^^^^^^^^
Hex 09 0d 0d 0a is \t\r\r\n.
Alternatively, you can print the lines with p and any invisible characters will be revealed.
f = File.open(file_name)
p f.readline
"0000001904\tABRAHAM SECURITIES CORPORATION\t00829452\t3724 47TH STREET CT. NW\t \tGIG HARBOR\tWA\t98335\t\r\r\n"
Use :row_sep => :auto instead of :row_sep => "\n\r":
CSV.open(file_name, "r", { :col_sep => "\t", :row_sep => :auto }).each do |row|
puts row
end

Bartender Webservice gives a BTXML Script & ACCESS_VIOLATION Error

We have setup a new Bartender instance in a new server due to the following BTXML error Web service integration and recently it was upgraded to BT 2019 R2 and it was working fine couple of days and now it showed the BTXML error.
But after the new installation now ACCESS_VIOLATION error is coming as shown below.
Any advises on how to fix it?
Following is the BTXML Script,
<XMLScript Version="2.0" Trusted="true">
<Command Name="Print Document">
<Print SaveAfterPrintCondition="IfModified" ReturnPrintData="false" ReturnSummary="false" ReturnLabelData="false" ReturnChecksum="false" ReturnDataSources="false">
<Format CloseAtEndOfJob="false" SaveAtEndOfJob="false" RegenerateThumbnail="false">C:\BT\CartonLabelMixed_small.btw</Format>
<PrintSetup>
<Printer>Generic_ZPL</Printer>
</PrintSetup>
<QueryPrompt Name="ID">
<Value>1000000008</Value>
</QueryPrompt>
</Print>
</Command>
</XMLScript>
Following is the ACCESS_VIOLATION ERROR,
Print job 'CartonLabelMixed_small.btw' did not complete due to the following error: BarTender Image Dump: Exception ACCESS_VIOLATION occurred at address 0x00000001400B6FFB inside module C:\Program Files\Seagull\BarTender 2019\BarTend.exe loaded with base address 0x0000000140000000. (x64 PID:6424)
Invalid read operation at address 0x0x0000000000000080
0: 00000001407C393D MethodBase:0000000000000000 !
1: 00000001407C3E88 MethodBase:0000000000000000 !
2: 00000001407C4393 MethodBase:0000000000000000 !
3: 000000005EDCF769 MethodBase:000000005EDCF5F4 MSVCR100.dll!_CxxFrameHandler3
4: 000000005EDD0BCC MethodBase:000000005EDD0850 MSVCR100.dll!_CxxExceptionFilter
5: 000000005EDD1235 MethodBase:000000005EDD0850 MSVCR100.dll!_CxxExceptionFilter
6: 000000005EDD144D MethodBase:000000005EDD0850 MSVCR100.dll!_CxxExceptionFilter
7: 000000005EDCF66B MethodBase:000000005EDCF5F4 MSVCR100.dll!_CxxFrameHandler3
8: 00007FFE4F2D992D MethodBase:00007FFE4F2D9810 ntdll.dll!_chkstk
9: 00007FFE4F2786D3 MethodBase:00007FFE4F278250 ntdll.dll!RtlImageNtHeaderEx
10: 00007FFE4F2D8A4A MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
11: 00000001400B6FFB MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
12: 00000001400B6DCC MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
13: 00000001400D4E26 MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
14: 00000001401E4066 MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
15: 00000001401CBE33 MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
16: 00000001400CF0E1 MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
17: 00000001400FAE16 MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
18: 00000001400CB67D MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
19: 00000001404202A4 MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
20: 0000000140462B99 MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
21: 0000000140452ADE MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
22: 00000001404510B6 MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
23: 0000000140452D2F MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
24: 0000000140450CD4 MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
25: 0000000140453B7E MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
26: 0000000140415276 MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
27: 0000000140415040 MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
28: 0000000140414E12 MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
29: 0000000140415352 MethodBase:00007FFE4F2D8A10 ntdll.dll!KiUserExceptionDispatcher
30: 00007FFE4E687DE3 MethodBase:00007FFE4E687670 RPCRT4.dll!NdrInterfacePointerMemorySize
31: 00007FFE4E6543EF MethodBase:00007FFE4E654060 RPCRT4.dll!NdrStubCall2
32: 00007FFE4E8E222F MethodBase:00007FFE4E8E21B0 combase.dll!CStdStubBuffer_Invoke
33: 00007FFE4C7E5535 MethodBase:00007FFE4C7E47B0 OLEAUT32.dll!VARIANT_UserMarshal
34: 00007FFE4E92DE3C MethodBase:00007FFE4E927400 combase.dll!Ordinal122
35: 00007FFE4E92E482 MethodBase:00007FFE4E927400 combase.dll!Ordinal122
36: 00007FFE4E945E58 MethodBase:00007FFE4E9431E0 combase.dll!CoGetCallContext
37: 00007FFE4E94450F MethodBase:00007FFE4E9431E0 combase.dll!CoGetCallContext
38: 00007FFE4E942584 MethodBase:00007FFE4E93A2E0 combase.dll!CoMarshalInterface
39: 00007FFE4E949206 MethodBase:00007FFE4E9431E0 combase.dll!CoGetCallContext
40: 00007FFE4E9498CA MethodBase:00007FFE4E9496C0 combase.dll!CoTaskMemFree
41: 00007FFE4CD91C24 MethodBase:00007FFE4CD91730 USER32.dll!CallWindowProcW
42: 00007FFE4CD9156C MethodBase:00007FFE4CD913B0 USER32.dll!DispatchMessageW
43: 000000005E734F5A MethodBase:00007FFE4CD913B0 USER32.dll!DispatchMessageW
44: 000000005E735823 MethodBase:00007FFE4CD913B0 USER32.dll!DispatchMessageW
45: 000000005E7674B0 MethodBase:00007FFE4CD913B0 USER32.dll!DispatchMessageW
46: 000000014083C676 MethodBase:00007FFE4CD913B0 USER32.dll!DispatchMessageW
47: 00007FFE0961BED1 MethodBase:00007FFE4CD913B0 USER32.dll!DispatchMessageW
48: 00007FFE39F56A53 MethodBase:00007FFE39F55E70 clr.dll!DllCanUnloadNowInternal
49: 00007FFE39F56913 MethodBase:00007FFE39F55E70 clr.dll!DllCanUnloadNowInternal
50: 00007FFE39F57165 MethodBase:00007FFE39F55E70 clr.dll!DllCanUnloadNowInternal
51: 00007FFE3A03D5CD MethodBase:00007FFE3A03C1C0 clr.dll!GetCLRFunction
52: 00007FFE3A03D426 MethodBase:00007FFE3A03C1C0 clr.dll!GetCLRFunction
53: 00007FFE3A03D2D6 MethodBase:00007FFE3A03C1C0 clr.dll!GetCLRFunction
54: 00007FFE3A03D784 MethodBase:00007FFE3A03C1C0 clr.dll!GetCLRFunction
55: 00007FFE3A03D702 MethodBase:00007FFE3A03C1C0 clr.dll!GetCLRFunction
56: 00007FFE3A03E154 MethodBase:00007FFE3A03E140 clr.dll!CorExeMain
57: 00007FFE3BD381AD MethodBase:00007FFE3BD38140 mscoreei.dll!CorExeMain
58: 00007FFE3C0C10AB MethodBase:00007FFE3C0C10A0 MSCOREE.DLL!CorExeMain
59: 00007FFE4EDA8364 MethodBase:00007FFE4EDA8350 KERNEL32.dll!BaseThreadInitThunk
60: 00007FFE4F295E91 MethodBase:00007FFE4F295E70 ntdll.dll!RtlUserThreadStart
Did you specify user credentials in the integration builder? In my experience BT is not good with using the " inherit from service" option ( under integration ==> User Account) Make sure that the account you use is one that has rights on the domain and include the domain with the user.
About the print engine, I know it sometimes has troubles when printing in large batches. Since this is not the case I also believe this is an issue with rights. There are other possibilities but I would start with the account.
Also make sure to stop the integration on the old server which had the print engine problem. Having two different integrations tugging on the same data to print can also cause some issues.

How to debug msgpack serialisation issue in Google Cloud Dataflow job?

I have a Google Cloud Dataflow job with which I would like to extract named entities from a text using a specific spacy model neural coref.
Running the extraction without beam I can extract entities but when I try to run it with the DirectRunner the job fails due to a serialisation error from msgpack. I am not sure how to proceed in debugging this problem.
My requirements are quite barebones with requirements of:
apache-beam[gcp]==2.4
spacy==2.0.12
ujson==1.35
The issue might be something related to how spacy and beam are interplaying as the stacktrace shows spacy spouting out some of its methods which it shouldn't be doing.
Weird spacy log behaviour from stacktrace:
T4: <class 'entity.extract_entities.EntityExtraction'>
# T4
D2: <dict object at 0x1126c0398>
T4: <class 'spacy.lang.en.English'>
# T4
D2: <dict object at 0x1126b54b0>
D2: <dict object at 0x1126d1168>
F2: <function is_alpha at 0x11266d320>
# F2
F2: <function is_ascii at 0x112327c08>
# F2
F2: <function is_digit at 0x11266d398>
# F2
F2: <function is_lower at 0x11266d410>
# F2
F2: <function is_punct at 0x112327b90>
# F2
F2: <function is_space at 0x11266d488>
# F2
F2: <function is_title at 0x11266d500>
# F2
F2: <function is_upper at 0x11266d578>
# F2
F2: <function like_url at 0x11266d050>
# F2
F2: <function like_num at 0x110d55140>
# F2
F2: <function like_email at 0x112327f50>
# F2
Fu: <functools.partial object at 0x11266c628>
F2: <function _create_ftype at 0x1070af500>
# F2
T1: <type 'functools.partial'>
F2: <function _load_type at 0x1070af398>
# F2
# T1
F2: <function is_stop at 0x11266d5f0>
# F2
D2: <dict object at 0x1126b7168>
T4: <type 'set'>
# T4
# D2
# Fu
F2: <function is_oov at 0x11266d668>
# F2
F2: <function is_bracket at 0x112327cf8>
# F2
F2: <function is_quote at 0x112327d70>
# F2
F2: <function is_left_punct at 0x112327de8>
# F2
F2: <function is_right_punct at 0x112327e60>
# F2
F2: <function is_currency at 0x112327ed8>
# F2
Fu: <functools.partial object at 0x110d49ba8>
F2: <function _get_attr_unless_lookup at 0x1106e26e0>
# F2
F2: <function lower at 0x11266d140>
# F2
D2: <dict object at 0x112317c58>
# D2
D2: <dict object at 0x110e38168>
# D2
D2: <dict object at 0x112669c58>
# D2
# Fu
F2: <function word_shape at 0x11266d0c8>
# F2
F2: <function prefix at 0x11266d1b8>
# F2
F2: <function suffix at 0x11266d230>
# F2
F2: <function get_prob at 0x11266d6e0>
# F2
F2: <function cluster at 0x11266d2a8>
# F2
F2: <function _return_en at 0x11266f0c8>
# F2
# D2
B2: <built-in function unpickle_vocab>
# B2
T4: <type 'spacy.strings.StringStore'>
# T4
My current hypothesis is that perhaps there is some problem with my setup.py but I am not sure what is causing the issue currently.
The full stacktrace is:
/Users/chris/coref_entity_extraction/venv/lib/python2.7/site-packages/msgpack_numpy.py:183: DeprecationWarning: encoding is deprecated, Use raw=False instead.
return _unpackb(packed, **kwargs)
/Users/chris/coref_entity_extraction/venv/lib/python2.7/site-packages/msgpack_numpy.py:132: DeprecationWarning: encoding is deprecated.
use_bin_type=use_bin_type)
T4: <class 'entity.extract_entities.EntityExtraction'>
# T4
D2: <dict object at 0x1126c0398>
T4: <class 'spacy.lang.en.English'>
# T4
D2: <dict object at 0x1126b54b0>
D2: <dict object at 0x1126d1168>
F2: <function is_alpha at 0x11266d320>
# F2
F2: <function is_ascii at 0x112327c08>
# F2
F2: <function is_digit at 0x11266d398>
# F2
F2: <function is_lower at 0x11266d410>
# F2
F2: <function is_punct at 0x112327b90>
# F2
F2: <function is_space at 0x11266d488>
# F2
F2: <function is_title at 0x11266d500>
# F2
F2: <function is_upper at 0x11266d578>
# F2
F2: <function like_url at 0x11266d050>
# F2
F2: <function like_num at 0x110d55140>
# F2
F2: <function like_email at 0x112327f50>
# F2
Fu: <functools.partial object at 0x11266c628>
F2: <function _create_ftype at 0x1070af500>
# F2
T1: <type 'functools.partial'>
F2: <function _load_type at 0x1070af398>
# F2
# T1
F2: <function is_stop at 0x11266d5f0>
# F2
D2: <dict object at 0x1126b7168>
T4: <type 'set'>
# T4
# D2
# Fu
F2: <function is_oov at 0x11266d668>
# F2
F2: <function is_bracket at 0x112327cf8>
# F2
F2: <function is_quote at 0x112327d70>
# F2
F2: <function is_left_punct at 0x112327de8>
# F2
F2: <function is_right_punct at 0x112327e60>
# F2
F2: <function is_currency at 0x112327ed8>
# F2
Fu: <functools.partial object at 0x110d49ba8>
F2: <function _get_attr_unless_lookup at 0x1106e26e0>
# F2
F2: <function lower at 0x11266d140>
# F2
D2: <dict object at 0x112317c58>
# D2
D2: <dict object at 0x110e38168>
# D2
D2: <dict object at 0x112669c58>
# D2
# Fu
F2: <function word_shape at 0x11266d0c8>
# F2
F2: <function prefix at 0x11266d1b8>
# F2
F2: <function suffix at 0x11266d230>
# F2
F2: <function get_prob at 0x11266d6e0>
# F2
F2: <function cluster at 0x11266d2a8>
# F2
F2: <function _return_en at 0x11266f0c8>
# F2
# D2
B2: <built-in function unpickle_vocab>
# B2
T4: <type 'spacy.strings.StringStore'>
# T4
Traceback (most recent call last):
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/Users/chris/coref_entity_extraction/main.py", line 29, in <module>
run()
File "/Users/chris/coref_entity_extraction/main.py", line 24, in run
entities = records | 'ExtractEntities' >> beam.ParDo(EntityExtraction())
File "/Users/chris/coref_entity_extraction/venv/lib/python2.7/site-packages/apache_beam/transforms/core.py", line 784, in __init__
super(ParDo, self).__init__(fn, *args, **kwargs)
File "/Users/chris/coref_entity_extraction/venv/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py", line 638, in __init__
self.fn = pickler.loads(pickler.dumps(self.fn))
File "/Users/chris/coref_entity_extraction/venv/lib/python2.7/site-packages/apache_beam/internal/pickler.py", line 204, in dumps
s = dill.dumps(o)
File "/Users/chris/coref_entity_extraction/venv/lib/python2.7/site-packages/dill/dill.py", line 259, in dumps
dump(obj, file, protocol, byref, fmode, recurse)#, strictio)
File "/Users/chris/coref_entity_extraction/venv/lib/python2.7/site-packages/dill/dill.py", line 252, in dump
pik.dump(obj)
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/pickle.py", line 425, in save_reduce
save(state)
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/Users/chris/coref_entity_extraction/venv/lib/python2.7/site-packages/apache_beam/internal/pickler.py", line 172, in new_save_module_dict
return old_save_module_dict(pickler, obj)
File "/Users/chris/coref_entity_extraction/venv/lib/python2.7/site-packages/dill/dill.py", line 841, in save_module_dict
StockPickler.save_dict(pickler, obj)
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/pickle.py", line 655, in save_dict
self._batch_setitems(obj.iteritems())
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/pickle.py", line 692, in _batch_setitems
save(v)
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/pickle.py", line 425, in save_reduce
save(state)
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/Users/chris/coref_entity_extraction/venv/lib/python2.7/site-packages/apache_beam/internal/pickler.py", line 172, in new_save_module_dict
return old_save_module_dict(pickler, obj)
File "/Users/chris/coref_entity_extraction/venv/lib/python2.7/site-packages/dill/dill.py", line 841, in save_module_dict
StockPickler.save_dict(pickler, obj)
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/pickle.py", line 655, in save_dict
self._batch_setitems(obj.iteritems())
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/pickle.py", line 687, in _batch_setitems
save(v)
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/pickle.py", line 401, in save_reduce
save(args)
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/pickle.py", line 568, in save_tuple
save(element)
File "/Users/chris/.pyenv/versions/2.7.14/lib/python2.7/pickle.py", line 306, in save
rv = reduce(self.proto)
File "vectors.pyx", line 108, in spacy.vectors.Vectors.__reduce__
File "vectors.pyx", line 409, in spacy.vectors.Vectors.to_bytes
File "/Users/chris/coref_entity_extraction/venv/lib/python2.7/site-packages/spacy/util.py", line 485, in to_bytes
serialized[key] = getter()
File "vectors.pyx", line 404, in spacy.vectors.Vectors.to_bytes.serialize_weights
File "/Users/chris/coref_entity_extraction/venv/lib/python2.7/site-packages/msgpack_numpy.py", line 165, in packb
return Packer(**kwargs).pack(o)
File "msgpack/_packer.pyx", line 282, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 288, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 285, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 232, in msgpack._cmsgpack.Packer._pack
File "msgpack/_packer.pyx", line 279, in msgpack._cmsgpack.Packer._pack
TypeError: can not serialize 'buffer' object
I have no idea about how to debug this issue with beam. To reproduce the whole issue I have setup a repo with instructions about how to set everything: https://github.com/swartchris8/coref_barebones
Are you able to run the same code from a regular Python program (not from a Beam DoFn) ?
If not, check whether you are storing any non-serializable state in a Beam DoFn (or any other function that will be serialized by Beam). This prevents Beam runners from serializing these functions (to be sent to workers) hence should be avoided.
In the end I got rid of the above the issue by changing the package versions installed. I do think it debugging the beam setup process is quite painful though my approach was just to manually try different package permutations.

MIPS Error in : invalid program counter value: 0x00000000

test_1:
addi $sp, $sp, -4
sw $ra, 0($sp)
jal test_2
lw $ra, 0($sp)
addi $sp, $sp, 4
jr $ra
test_2:
addi $sp, $sp, -4
sw $ra, 4($sp)
jal test_3
lw $ra, 4($sp)
addi $sp, $sp, 4
jr $ra
Error: Error in : invalid program counter value: 0x00000000. Go: execution terminated with errors.
I don't understand why I get this error. My understanding is that test_1 makes space for one in the stack and saves its $ra. After the jal test_2 instruction we jump to test_2 and here test_2 makes space for the second $ra (-4 -4 = -8).
What am I doing wrong?

Weird issue with i18n rails haml

I was looking into How to use rails-i18n with HAML to find out how i18n works together with haml but ran into an issue I can't figure out.
This works:
en.yml
en:
sitename: "Happy Sunday"
new.haml
%h1= t("sitename")
When I change the yml to
en.yml
en:
home:
sitename: "Happy Sunday"
new.haml
%h1= t("home.sitename")
Then I get the following error:
ArgumentError in Devise/sessions#new
Showing
..../devise/sessions/new.html.haml where line #20 raised:
syntax error on line 4, col 6: ` home:'
Extracted source (around line #20):
17: = flash[:alert]
18: .row
19: .headline.pagination-centered
20: %h1= t("home.sitename")
21: %h2= t("slogan")
22: .row.headline.pagination-centered
23: %a{:href => "/tour"}
The message:
syntax error on line 4, col 6: ` home:'
suggests an error in your Yaml. Check en.yml, especially that you’re not using tabs and that your indentation is consistent.

Resources