MAIL2 TPOP3d TROUBLE INFORMATION All decimal figures are percentages SCPU = Total cpu usage on the system SMEM = Total memory usage on the system Memfree = Total real memory free on the system (out of 512mb) SwpFree = Total swap space free on the system (out of 1024mb) TpCPU = Total CPU used by all Tpop processes combined TpMem = Total memory used by all Tpop processed combined TMAXCPU = CPU used by the parent Tpop process TMAXMEM = Memory used by the parent Tpop process #Tpop = Total number of Tpop processes running Time SCPU SMEM MemFree SwpFree TpCPU TpMEM TMAXCPU TMAXMEM #Tpop 01/08-20:00:01 14.2 38.3 192112 656884 9.5 6.9 8.5 4.5 7 01/08-20:00:10 13.7 38.6 191808 636264 9.8 7.0 8.2 4.5 7 01/08-20:10:11 13.6 36.4 184468 636132 10.4 6.9 8.7 4.7 8 01/08-20:15:39 10.9 37.2 184300 665652 6.9 6.6 6.4 4.9 6 01/08-20:20:10 12.6 40.2 181348 568980 8.7 7.3 7.5 4.9 9 01/08-20:30:10 10.5 35.7 187228 602696 8.1 6.9 7.3 5.1 7 01/08-20:40:11 10.6 38.0 179352 681012 7.7 7.3 7.0 5.2 7 01/08-20:50:10 9.6 37.7 184200 645388 5.9 7.1 5.1 5.4 6 01/08-21:00:11 12.8 38.8 168128 558260 8.6 8.4 7.5 5.6 9 01/08-21:10:10 11.3 38.7 166652 628328 7.3 8.2 6.6 5.9 5 01/08-21:20:10 10.1 34.9 164980 644124 7.2 7.5 6.6 5.9 6 01/08-21:30:11 9.5 44.9 158872 594612 6.2 7.9 5.1 6.1 5 01/08-21:40:10 9.6 40.5 170060 585128 6.2 8.4 4.9 6.3 7 01/08-21:50:10 11.8 35.3 170828 523624 8.7 9.3 7.2 6.4 9 01/08-22:00:11 8.0 37.4 162636 512904 6.0 8.9 4.9 6.6 8 01/08-22:10:10 8.3 37.8 169872 533240 5.7 9.7 4.6 6.7 7 01/08-22:20:10 7.9 37.2 172984 651732 5.5 7.8 4.7 6.8 4 01/08-22:30:11 8.4 37.5 169696 686824 4.9 7.5 4.5 6.9 3 01/08-22:40:10 7.0 36.9 168120 681344 4.9 7.9 4.0 7.0 4 01/08-22:50:10 9.7 43.6 169604 684668 6.0 7.8 5.9 7.2 3 01/08-23:00:11 8.1 40.5 181988 697876 5.0 8.2 4.6 7.3 4 01/08-23:10:10 4.9 35.5 178612 681648 2.8 8.2 2.6 7.4 3 01/08-23:20:10 10.6 41.0 174448 711908 5.8 7.8 5.7 7.5 2 01/08-23:30:11 6.7 39.1 174632 746076 4.2 7.9 4.2 7.6 2 01/08-23:40:10 5.1 35.8 180316 645140 3.4 8.6 2.8 7.7 4 01/08-23:50:10 5.8 35.1 174336 681360 3.6 8.3 3.3 7.7 3 01/09-00:00:11 6.2 35.2 168900 708120 3.8 8.8 3.5 7.8 3 01/09-00:10:11 5.5 36.9 187528 601696 3.8 10.0 3.2 7.9 5 01/09-00:20:10 5.8 37.0 167948 590312 4.1 10.5 3.3 8.0 5 01/09-00:30:10 7.0 34.5 163948 727204 4.7 8.4 4.7 8.0 2 01/09-00:40:11 7.4 46.6 158784 723600 3.9 8.1 3.9 8.1 1 01/09-00:50:10 6.2 36.3 182616 718752 3.8 8.6 3.3 8.2 2 01/09-01:00:10 5.9 36.5 181744 720708 3.8 8.7 3.3 8.3 2 01/09-01:10:11 5.5 35.0 177300 687732 3.7 9.0 2.9 8.3 3 01/09-01:20:10 6.8 39.7 174312 631028 3.9 9.7 2.5 8.4 5 01/09-01:30:10 5.1 35.3 190372 685136 2.6 9.2 2.2 8.5 3 01/09-01:40:10 4.9 37.5 188992 647060 2.5 9.2 2.2 8.5 3 01/09-01:50:10 5.9 37.0 188772 672844 3.9 9.6 2.9 8.6 4 01/09-02:00:10 5.5 36.9 189584 738576 3.2 8.7 3.2 8.7 1 01/09-02:10:10 6.8 41.2 188248 748612 3.3 8.8 3.3 8.8 1 01/09-02:20:11 5.1 36.5 194752 742684 3.3 9.2 3.3 8.9 2 01/09-02:30:10 5.0 40.3 194016 686640 3.2 9.5 3.0 8.9 3 01/09-02:40:10 5.1 38.5 198156 709084 2.9 9.6 2.8 9.0 3 01/09-02:50:11 5.9 35.8 199852 690388 3.1 10.0 2.9 9.1 4 01/09-03:00:10 5.3 39.9 196524 653896 3.6 9.7 3.4 9.1 3 01/09-03:10:10 6.8 37.5 197820 650904 4.6 9.5 4.1 9.2 2 01/09-03:20:11 5.9 40.4 198528 738272 3.1 9.3 3.1 9.3 1 01/09-03:30:10 4.9 35.8 200912 743276 2.9 9.3 2.9 9.3 1 01/09-03:40:10 5.9 40.6 201296 728648 3.5 9.4 3.5 9.4 1 01/09-03:50:11 4.6 40.4 199164 691540 2.0 9.8 1.9 9.5 2 01/09-04:00:10 5.7 38.3 200844 752520 2.6 9.5 2.6 9.5 1 01/09-04:10:10 5.6 37.0 203016 677836 3.0 10.4 2.7 9.6 4 01/09-04:20:10 5.4 31.3 212072 701600 3.5 10.3 3.4 9.7 3 01/09-04:30:11 4.0 29.9 219744 744828 2.2 10.2 2.1 9.7 3 01/09-04:40:10 4.9 31.6 220208 656048 3.5 11.0 3.3 9.8 5 01/09-04:50:10 4.9 35.6 218636 745980 3.1 10.2 2.9 9.9 2 01/09-05:00:11 3.7 35.1 219452 758312 2.4 10.9 2.2 10.1 3 01/09-05:10:10 5.6 34.1 225532 750068 3.3 10.1 3.3 10.1 1 01/09-05:20:10 5.2 33.6 227940 709116 3.3 10.2 3.3 10.2 1 01/09-05:30:10 4.2 37.3 231456 646508 2.9 12.3 2.5 10.3 6 01/09-05:40:11 6.4 36.0 232484 748460 3.6 10.3 3.6 10.3 1 01/09-05:50:10 6.9 36.6 228532 727636 3.8 10.4 3.8 10.4 1 01/09-06:00:10 5.4 39.3 226628 640312 3.3 11.4 3.2 10.5 3 01/09-06:10:11 8.3 39.2 225308 602504 5.0 12.2 4.6 10.6 5 01/09-06:20:10 7.2 36.7 230868 650108 5.2 11.5 5.0 10.7 3 01/09-06:30:10 6.2 38.7 224344 612556 4.4 11.2 3.9 10.8 2 01/09-06:40:11 6.1 38.7 218124 594140 4.2 12.2 3.4 10.9 4 01/09-06:50:10 6.9 42.6 213300 650732 3.9 11.4 3.9 11.0 2 01/09-07:00:10 7.6 43.5 213532 512452 4.4 12.8 3.4 11.1 5 01/09-07:10:11 9.9 43.6 211040 505692 6.1 12.9 4.8 11.3 5 01/09-07:20:10 8.7 43.2 206624 450464 6.2 14.3 5.3 11.3 7 01/09-07:30:10 8.3 41.8 184928 413116 6.7 15.6 4.4 11.5 7 01/09-07:40:11 12.7 48.4 166836 477312 9.6 17.2 6.5 11.6 8 01/09-07:50:10 9.0 41.4 160088 458252 6.8 13.9 5.6 11.7 6 01/09-08:00:11 12.7 45.2 154256 343844 9.2 14.0 8.5 11.9 6 01/09-08:10:10 12.2 50.2 149908 435616 7.5 13.8 6.8 12.0 5 At 8:11, Tpop reported "fork_child: fork: Not enough space" but continued to run and answer pop3 requests: 01/09-08:20:10 8.5 39.7 153060 511348 6.2 13.8 5.8 12.1 5 At 8:26, Tpop reported the error again, but then permanently stopped responding to requests. It was 16 hours since the last failure: Maildir Scans: 45,170 Total bytes received: 2,439,782 Total bytes sent: 533,903,104 Before that, it had been 7 hours since the last failure. During that 7 hours, it was our first time with tpop3d in production, so many people had to download previously saved email: Maildir Scans: 47,115 Total bytes received: 2,673,760 Total bytes sent: 739,178,518