[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Gluster-devel] Language Bindings and Data Serialization in Gluster
From: |
Brad Childs |
Subject: |
[Gluster-devel] Language Bindings and Data Serialization in Gluster |
Date: |
Fri, 15 Feb 2013 00:46:13 -0500 (EST) |
I'm working on a hadoop/gluster integration. First, whoever wrote the initial
map/reduce gluster DFS layer did a great job. Same with the FUSE layer-- FUSE
alone solves a lot of problems.
However, some of the hadoop architectures require cross language (java)
bindings with gluster (c). Other's I've talked to about cross language bindings
have shown interest in gluster--> python and ruby bindings. From the research
I've done, most of a common language interface is simple and the only tricky
part is data serialization.
I'm not an expert in data serialization across programming languages so it's
been a lot of research. There's a few new and old solutions to the problem but
I think the older non IDL solutions are obvious and not really that interesting
(RPC, ASN.1, raw JNI or other language specific native access).
Moving up to IDL abstractions, there's Thrift, Protobuf and SWIG. SWIG is
really good for simple multi-language bindings but requires manual data
serialization.
>From my research the common solutions for multi-language serialization today
>is either protobuf or thrift. Thrift is a facebook originated solution (now
>under apache) and protobuf is google's open solution.
I'm leaning towards thrift since it's apache. A few of the hadoop
architectures already support thrift so that would simplify things. What's
your opinion? Is anyone familiar with, or already solved I/O serialization
across programming languages in gluster? I'd love some feedback and discussion.
Links:
SWIG: http://www.swig.org/
Protobuf: http://code.google.com/p/protobuf/
Serialization Discussion:
http://stackoverflow.com/questions/4633611/what-are-the-key-differences-between-apache-thrift-google-protocol-buffers-mes
Previous Gluster Interface Pitch:
https://fedoraproject.org/wiki/Summer_coding_ideas_for_2012#Implement_a_binding_translator_for_GlusterFS
Humor: http://www.quickmeme.com/meme/3t05wa/
Thanks!
bradley childs
- Re: [Gluster-devel] regressions due to 64-bit ext4 directory cookies, (continued)
- Re: [Gluster-devel] regressions due to 64-bit ext4 directory cookies, Theodore Ts'o, 2013/02/13
- Re: [Gluster-devel] regressions due to 64-bit ext4 directory cookies, Anand Avati, 2013/02/13
- Re: [Gluster-devel] regressions due to 64-bit ext4 directory cookies, J. Bruce Fields, 2013/02/13
- Re: [Gluster-devel] regressions due to 64-bit ext4 directory cookies, Theodore Ts'o, 2013/02/13
- Re: [Gluster-devel] regressions due to 64-bit ext4 directory cookies, Anand Avati, 2013/02/13
- Re: [Gluster-devel] regressions due to 64-bit ext4 directory cookies, J. Bruce Fields, 2013/02/14
- Re: [Gluster-devel] regressions due to 64-bit ext4 directory cookies, J. Bruce Fields, 2013/02/14
- Re: [Gluster-devel] regressions due to 64-bit ext4 directory cookies, Dave Chinner, 2013/02/14
- Re: [Gluster-devel] regressions due to 64-bit ext4 directory cookies, J. Bruce Fields, 2013/02/14
- Re: [Gluster-devel] regressions due to 64-bit ext4 directory cookies, Dave Chinner, 2013/02/14
- [Gluster-devel] Language Bindings and Data Serialization in Gluster,
Brad Childs <=
- Re: [Gluster-devel] Language Bindings and Data Serialization in Gluster, Jay Vyas, 2013/02/15
Message not available
Re: [Gluster-devel] regressions due to 64-bit ext4 directory cookies, John Mark Walker, 2013/02/13