[HILTI] HILTI Digest, Vol 83, Issue 3
Fernandez, Mark I
mfernandez at mitre.org
Mon Jul 31 02:42:39 PDT 2017
Jennifer,
You can find an example in the HTTP spicy parser:
https://github.com/rsmmr/hilti/blob/master/libspicy/parsers/http.spicy
See lines 80-83; line 157; and lines 175-187.
I hope this helps.
Mark
From: hilti-bounces at ICSI.Berkeley.EDU [mailto:hilti-bounces at ICSI.Berkeley.EDU] On Behalf Of Jennifer Gates
Sent: Saturday, July 29, 2017 10:09 PM
To: hilti at ICSI.Berkeley.EDU
Subject: Re: [HILTI] HILTI Digest, Vol 83, Issue 3
Is there a good example you can point me to for passing the unit to the sub unit as you suggest? Just adding self in the connect line gives me the following error:
>>> (self)
<no location>:: error, too many arguments [pass::spicy::Validator]
Thank you,
Jenn
On Fri, Jul 28, 2017 at 3:00 PM, <hilti-request at icsi.berkeley.edu<mailto:hilti-request at icsi.berkeley.edu>> wrote:
Send HILTI mailing list submissions to
hilti at mailman.ICSI.Berkeley.EDU<mailto:hilti at mailman.ICSI.Berkeley.EDU>
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/hilti
or, via email, send a message with subject or body 'help' to
hilti-request at mailman.ICSI.Berkeley.EDU<mailto:hilti-request at mailman.ICSI.Berkeley.EDU>
You can reach the person managing the list at
hilti-owner at mailman.ICSI.Berkeley.EDU<mailto:hilti-owner at mailman.ICSI.Berkeley.EDU>
When replying, please edit your Subject line so it is more specific
than "Re: Contents of HILTI digest..."
Today's Topics:
1. Re: Spicy Parsing WebSockets - how to end a list
(Fernandez, Mark I)
2. Re: Spicy Parsing WebSockets - how to end a list (Jennifer Gates)
3. Re: Spicy Parsing WebSockets - how to end a list
(Fernandez, Mark I)
4. Re: Spicy Parsing WebSockets - how to end a list (Robin Sommer)
----------------------------------------------------------------------
Message: 1
Date: Thu, 27 Jul 2017 19:09:08 +0000
From: "Fernandez, Mark I" <mfernandez at mitre.org<mailto:mfernandez at mitre.org>>
Subject: Re: [HILTI] Spicy Parsing WebSockets - how to end a list
To: Jennifer Gates <jenngates2004 at gmail.com<mailto:jenngates2004 at gmail.com>>, Robin Sommer
<robin at icir.org<mailto:robin at icir.org>>
Cc: "hilti at ICSI.Berkeley.EDU<mailto:hilti at ICSI.Berkeley.EDU>" <hilti at ICSI.Berkeley.EDU<mailto:hilti at ICSI.Berkeley.EDU>>
Message-ID:
<MWHPR09MB1533E7FD664492267457EEE8CFBE0 at MWHPR09MB1533.namprd09.prod.outlook.com<mailto:MWHPR09MB1533E7FD664492267457EEE8CFBE0 at MWHPR09MB1533.namprd09.prod.outlook.com>>
Content-Type: text/plain; charset="utf-8"
Is the sink executing as you intend? If not, it could be because when the hook ?on %init? is executed when the routine is initialized, but at that time, the element ?wss_data? has not been parsed yet. Perhaps try:
on wss_data {
self.sub.connect(new Sub());
self.sub.write(self.wss_data);
}
Mark
From: hilti-bounces at ICSI.Berkeley.EDU<mailto:hilti-bounces at ICSI.Berkeley.EDU> [mailto:hilti-bounces at ICSI.Berkeley.EDU<mailto:hilti-bounces at ICSI.Berkeley.EDU>] On Behalf Of Jennifer Gates
Sent: Thursday, July 27, 2017 2:56 PM
To: Robin Sommer <robin at icir.org<mailto:robin at icir.org>>
Cc: hilti at ICSI.Berkeley.EDU<mailto:hilti at ICSI.Berkeley.EDU>
Subject: Re: [HILTI] Spicy Parsing WebSockets - how to end a list
Thanks for the response. Unfortunately, the &eod doesn't work with a list, just with bytes. Because the end of the connection is simply the end of the last byte of actual data, there is no look-ahead token for the list to work. At least that is what it seems to me.
Last night I started down another path. I am trying to use a sink construct. Here is some of the code:
export type WS_Handshake_Success = unit {
success : /^(HTTP|http)/;
dvalue : DataValue;
svrheaders : list<Header> ;
end_of_hdrs : /\x0d\x0a\x0d\x0a/;
wss_data : bytes &eod &convert=terminate($$) &transient -> self.sub;
on %init {
self.sub.connect(new Sub());
}
var sub: sink;
};
export type Sub = unit {
ws_msgs : list<WS_Message>;
: DataEnd;
};
The terminate function appends a byte sequence (constant DataEnd) to the raw bytes that I can then use to identify the end of the data when parsing the messages in the Sub unit. That way the WS_Messages are properly parsed within the Sub unit.
The only piece that I'm working on now is how to put that parsed data back into the original WS_Handshake unit. Right now a "print self" statement of the WS_Handshake just lists sub=<sink>. To get the individual list items to show I need to print the sub unit.
Is there a way to make the Sub unit a part of the WS_Handshake unit?
Thank you,
Jenn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/hilti/attachments/20170727/c35009df/attachment-0001.html
------------------------------
Message: 2
Date: Thu, 27 Jul 2017 15:23:16 -0400
From: Jennifer Gates <jenngates2004 at gmail.com<mailto:jenngates2004 at gmail.com>>
Subject: Re: [HILTI] Spicy Parsing WebSockets - how to end a list
To: "Fernandez, Mark I" <mfernandez at mitre.org<mailto:mfernandez at mitre.org>>
Cc: "hilti at ICSI.Berkeley.EDU<mailto:hilti at ICSI.Berkeley.EDU>" <hilti at ICSI.Berkeley.EDU<mailto:hilti at ICSI.Berkeley.EDU>>
Message-ID:
<CAMV56f0hT2-j3TskcZvrSTUQ4RfpVLX1jNcwQAh_VBoOX7hGDw at mail.gmail.com<mailto:CAMV56f0hT2-j3TskcZvrSTUQ4RfpVLX1jNcwQAh_VBoOX7hGDw at mail.gmail.com>>
Content-Type: text/plain; charset="utf-8"
Hi,
I thought the sink would still keep the sub data as part of the original
WS_Handshake unit and not a separate new unit. Otherwise, it does appear to
be doing what I intended. Changing my code to how you suggested produces
the same results.
I guess I'm just wondering if there is a way to have the sink data come
back to the WS_Handshake unit after it is parsed by the Sub and WS_Messages
units? (I'm thinking it can't since it is already parsed.) Or is there away
to add the sub unit's ws_msgs field as a variable within the WS_Handshake
unit, or somehow append the sub unit to the WS_Handshake unit?
Thank you,
Jenn
Jenn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/hilti/attachments/20170727/19d2a046/attachment-0001.html
------------------------------
Message: 3
Date: Thu, 27 Jul 2017 19:54:52 +0000
From: "Fernandez, Mark I" <mfernandez at mitre.org<mailto:mfernandez at mitre.org>>
Subject: Re: [HILTI] Spicy Parsing WebSockets - how to end a list
To: Jennifer Gates <jenngates2004 at gmail.com<mailto:jenngates2004 at gmail.com>>
Cc: "hilti at ICSI.Berkeley.EDU<mailto:hilti at ICSI.Berkeley.EDU>" <hilti at ICSI.Berkeley.EDU<mailto:hilti at ICSI.Berkeley.EDU>>
Message-ID:
<MWHPR09MB15339E3878A0186204FD0655CFBE0 at MWHPR09MB1533.namprd09.prod.outlook.com<mailto:MWHPR09MB15339E3878A0186204FD0655CFBE0 at MWHPR09MB1533.namprd09.prod.outlook.com>>
Content-Type: text/plain; charset="utf-8"
>> if there is a way to have the sink data come back to the WS_Handshake
>> unit after it is parsed by the Sub and WS_Messages units? (I'm thinking
>> it can't since it is already parsed.)
I encountered this dilemma too, but did not find a way around it. In WS_Handhake, you can reference self.wss_data, but it will be just a byte array, will not have structure or format to it. That is my experience, but I am new to spicy, don?t know all the tips or tricks.
Mark
From: Jennifer Gates [mailto:jenngates2004 at gmail.com<mailto:jenngates2004 at gmail.com>]
Sent: Thursday, July 27, 2017 3:23 PM
To: Fernandez, Mark I <mfernandez at mitre.org<mailto:mfernandez at mitre.org>>
Cc: hilti at ICSI.Berkeley.EDU<mailto:hilti at ICSI.Berkeley.EDU>
Subject: Re: [HILTI] Spicy Parsing WebSockets - how to end a list
Hi,
I thought the sink would still keep the sub data as part of the original WS_Handshake unit and not a separate new unit. Otherwise, it does appear to be doing what I intended. Changing my code to how you suggested produces the same results.
I guess I'm just wondering if there is a way to have the sink data come back to the WS_Handshake unit after it is parsed by the Sub and WS_Messages units? (I'm thinking it can't since it is already parsed.) Or is there away to add the sub unit's ws_msgs field as a variable within the WS_Handshake unit, or somehow append the sub unit to the WS_Handshake unit?
Thank you,
Jenn
Jenn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/hilti/attachments/20170727/8b9e7e78/attachment-0001.html
------------------------------
Message: 4
Date: Fri, 28 Jul 2017 08:21:06 -0700
From: Robin Sommer <robin at icir.org<mailto:robin at icir.org>>
Subject: Re: [HILTI] Spicy Parsing WebSockets - how to end a list
To: Jennifer Gates <jenngates2004 at gmail.com<mailto:jenngates2004 at gmail.com>>, "Fernandez, Mark I"
<mfernandez at mitre.org<mailto:mfernandez at mitre.org>>
Cc: hilti at ICSI.Berkeley.EDU<mailto:hilti at ICSI.Berkeley.EDU>
Message-ID: <20170728152106.GQ52171 at icir.org<mailto:20170728152106.GQ52171 at icir.org>>
Content-Type: text/plain; charset=us-ascii
On Thu, Jul 27, 2017 at 14:55 -0400, Jennifer wrote:
> Thanks for the response. Unfortunately, the &eod doesn't work with a list,
> just with bytes.
Ah, of course, that's right for the attribute. However, thinking about
it, lists should actually be implicitly terminated by the end of the
input, even without any attribute. There's a test for that in
tests/spicy/list/eof.spicy:
module Test;
export type test = unit {
: list<b> &transient;
};
type b = unit {
a: uint8 {
print self.a;
}
};
# echo -n 1234 | spicy-driver tests/spicy/list/eof.spicy
49
50
51
52
(The &transient doesn't make a difference.)
Are you getting an error message about a missing look-ahead token when
trying this with your list? If you send me the full Spicy code, I'll
see if I can figure out why, it might be that the parser generator
doesn't quite recognize the situation.
> Last night I started down another path. I am trying to use a sink
> construct.
Nice idea!
> Is there a way to make the Sub unit a part of the WS_Handshake unit?
If you pass the handshake unit to the new Sub unit
("self.sub.connect(new Sub(self))"), you can then use that inside Sub
to assign to variables stored inside the handshake unit, and bring the
information back that way. Does that help?
Robin
--
Robin Sommer * ICSI/LBNL * robin at icir.org<mailto:robin at icir.org> * www.icir.org/robin<http://www.icir.org/robin>
------------------------------
_______________________________________________
HILTI mailing list
HILTI at mailman.ICSI.Berkeley.EDU<mailto:HILTI at mailman.ICSI.Berkeley.EDU>
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/hilti
End of HILTI Digest, Vol 83, Issue 3
************************************
--
Jenn
---------------------------------------------------
email: jenngates2004 at gmail.com<mailto:jenngates2004 at gmail.com>
blog: https://collateralmeaning.blogspot.com
twitter: @jgsecure
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/hilti/attachments/20170731/11a6b877/attachment.html
More information about the HILTI
mailing list