Attribute on streams
A common operation on a stream is to add an attribute.
For example, you might need to add a time stamp to a
stream.
type streamType = int32 a, int32 b;
streamTypeFull = int32 a, int32 b, timestamp t;
graph
stream<streamType> src = SomeSource() {
....
}
stream<streamTypeFull> tgt = Functor(src) {
output tgt : t = getTimestamp();
}
The Functor
operator copies all
attributes from the input stream into the output stream and adds the time
stamp. This processing might be expensive when the stream contains
many attributes. The copy is more noticeable when the two operators
are fused in the same PE.
In general, a more efficient way to add an attribute to a stream is to preallocate the attribute
in an upstream operator, and then set the value of that attribute by using a
Custom operator instead of a Functor operator. Using the time
stamp example, here is a more efficient way to add an attribute to a
stream.
stream<streamTypeFull> src = SomeSource() {
....
}
stream<streamTypeFull> tgt = Custom(src) {
logic onTuple src: {
t = getTimestamp(); // update input stream
submit (src,tgt);
}
}
The
attribute is preallocated by the SomeSource operator. It is not always possible
or easy to preallocate an attribute. For example, the FileSource operator expects
that all attributes are to be read from a file.