[Gluster-devel] gluster working, but error appearing every two seconds in logs

[Jordi Moles Blanco]

Hello everyone,

I'm using gluster 1.3.x version, patch 800 from tla repositories.

The thing is that i have 6 nodes, providing a total amount of 2TB and several 
clients accessing the data constantly and putting gluster under a lot of work.

During several days i didn't pay attention to the gluster logs, as everything 
worked fine. However, today i decided i was moving a file sized 500MB and the 
mount point got stale, i couln't access the data from that particular client. 
The gluster itself didn't seem to be affected, nodes didn't report any problem 
at all in the log files and other clients kept the mount point without any 
problem.
Then i decided to have a look at the log files:


*************
2009-01-30 11:00:41 W [client-protocol.c:332:client_protocol_xfer] espai1: not 
connected at the moment to submit frame type(1) op(15)
2009-01-30 11:00:41 E [client-protocol.c:3891:client_statfs_cbk] espai1: no 
proper reply from server, returning ENOTCONN
2009-01-30 11:00:41 E [tcp-client.c:190:tcp_connect] espai5: non-blocking 
connect() returned: 111 (Connection refused)

2009-01-30 11:00:43 W [client-protocol.c:332:client_protocol_xfer] espai2: not 
connected at the moment to submit frame type(1) op(15)
2009-01-30 11:00:43 E [client-protocol.c:3891:client_statfs_cbk] espai2: no 
proper reply from server, returning ENOTCONN
2009-01-30 11:00:43 E [tcp-client.c:190:tcp_connect] espai6: non-blocking 
connect() returned: 111 (Connection refused)
*************

This goes on and on for days, and it prints some error message every 2-3 
seconds.

Is there any major bug in the version i'm using? Is there any way to fix this?

If i look through the whole logfile, i can't see any message (a part from this 
one that repeats every 2-3 seconds) which indicates why the mountpoint got 
stale and the data not accessible from that client.

Does this error message have anything to do with today's issue? Could that 
message cause a failure in the system when moving, deleting or creating large 
files?

This are the config files:



NODE:

***********

volume esp
       type storage/posix
       option directory /glu0/data
end-volume

volume espai
       type performance/io-threads
       option thread-count 15
       option cache-size 512MB
       subvolumes esp
end-volume

volume nm
       type storage/posix
       option directory /glu0/ns
end-volume

volume ultim
   type protocol/server
   subvolumes espai nm
   option transport-type tcp/server
   option auth.ip.espai.allow *
   option auth.ip.nm.allow *
end-volume

***********

CLIENT:

********
volume espai1
       type protocol/client
       option transport-type tcp/client
       option remote-host 10.0.0.3
       option remote-subvolume espai
end-volume

volume espai2
       type protocol/client
       option transport-type tcp/client
       option remote-host 10.0.0.4
       option remote-subvolume espai
end-volume

volume espai3
       type protocol/client
       option transport-type tcp/client
       option remote-host 10.0.0.5
       option remote-subvolume espai
end-volume

volume espai4
   type protocol/client
   option transport-type tcp/client
   option remote-host 10.0.0.6
   option remote-subvolume espai
end-volume

volume espai5
   type protocol/client
   option transport-type tcp/client
   option remote-host 10.0.0.7
   option remote-subvolume espai
end-volume

volume espai6
   type protocol/client
   option transport-type tcp/client
   option remote-host 10.0.0.8
   option remote-subvolume espai
end-volume

volume namespace1
       type protocol/client
       option transport-type tcp/client
       option remote-host 10.0.0.3
       option remote-subvolume nm
end-volume

volume namespace2
       type protocol/client
       option transport-type tcp/client
       option remote-host 10.0.0.4
       option remote-subvolume nm
end-volume

volume namespace3
       type protocol/client
       option transport-type tcp/client
       option remote-host 10.0.0.5
       option remote-subvolume nm
end-volume

volume namespace4
       type protocol/client
       option transport-type tcp/client
       option remote-host 10.0.0.6
       option remote-subvolume nm
end-volume

volume namespace5
       type protocol/client
       option transport-type tcp/client
       option remote-host 10.0.0.7
       option remote-subvolume nm
end-volume

volume namespace6
       type protocol/client
       option transport-type tcp/client
       option remote-host 10.0.0.8
       option remote-subvolume nm
end-volume

volume grup1
       type cluster/afr
       subvolumes espai1 espai3 espai5
end-volume

volume grup2
       type cluster/afr
       subvolumes espai2 espai4 espai6
end-volume

volume nm
       type cluster/afr
       subvolumes namespace1 namespace2 namespace3 namespace4 namespace5 
namespace6
end-volume

volume g01
       type cluster/unify
       subvolumes grup1 grup2
       option scheduler rr
       option namespace nm
end-volume

volume io-cache 
       type performance/io-cache 
       option cache-size 512MB 
       option page-size 1MB
       option force-revalidate-timeout 2 
       subvolumes g01
end-volume   

************

Thanks.