[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: map-par slower than map
From: |
Olivier Dion |
Subject: |
Re: map-par slower than map |
Date: |
Wed, 12 Oct 2022 17:55:10 -0400 |
On Wed, 12 Oct 2022, Damien Mattei <damien.mattei@gmail.com> wrote:
> Hello,
> all is in the title, i test on a approximately 30000 element list , i got
> 9s with map and 3min 30s with par-map on exactly the same piece of
> code!?
I can only speculate here. But trying with a very simple example here:
--8<---------------cut here---------------start------------->8---
(use-modules (statprof))
(statprof (lambda () (par-map 1+ (iota 300000))))
--8<---------------cut here---------------end--------------->8---
Performance are terrible. I don't know how par-map is implemented, but
if it does 1 element static scheduling -- which it probably does because
you pass a linked list and not a vector -- then yeah you can assure that
thing will be very slow.
You're probably better off with dynamic scheduling with vectors. Here's
a quick snippet I made for static scheduling but with vectors. Feel
free to roll your own.
--8<---------------cut here---------------start------------->8---
(use-modules
(srfi srfi-1)
(ice-9 threads))
(define* (par-map-vector proc input
#:optional
(max-thread (current-processor-count)))
(let* ((block-size (quotient (vector-length input) max-thread))
(rest (remainder (vector-length input) max-thread))
(output (make-vector (vector-length input) #f)))
(when (not (zero? block-size))
(let ((mtx (make-mutex))
(cnd (make-condition-variable))
(n 0))
(fold
(lambda (scale output)
(begin-thread
(let lp ((i 0))
(when (< i block-size)
(let ((i (+ i (* scale block-size))))
(vector-set! output i (proc (vector-ref input i))))
(lp (1+ i))))
(with-mutex mtx
(set! n (1+ n))
(signal-condition-variable cnd)))
output)
output
(iota max-thread))
(with-mutex mtx
(while (not (< n max-thread))
(wait-condition-variable cnd mtx))))
(let ((base (- (vector-length input) rest)))
(let lp ((i 0))
(when (< i rest)
(let ((i (+ i base)))
(vector-set! output i (proc (vector-ref input i))))
(lp (1+ i)))))
output))
--8<---------------cut here---------------end--------------->8---
--
Olivier Dion
oldiob.dev
- Re: map-par slower than map, (continued)
- Re: map-par slower than map, Damien Mattei, 2022/10/22
- Re: map-par slower than map, Damien Mattei, 2022/10/22
- Re: map-par slower than map, Zelphir Kaltstahl, 2022/10/23
- Re: map-par slower than map, Keith Wright, 2022/10/23
- Re: map-par slower than map, Zelphir Kaltstahl, 2022/10/24
- Re: map-par slower than map, Damien Mattei, 2022/10/24
- Re: map-par slower than map, Mikael Djurfeldt, 2022/10/25
- Re: map-par slower than map, Mikael Djurfeldt, 2022/10/25
- Re: map-par slower than map, Damien Mattei, 2022/10/25
- Re: map-par slower than map, Maxime Devos, 2022/10/12
Re: map-par slower than map,
Olivier Dion <=
- Re: map-par slower than map, Damien Mattei, 2022/10/13
- Re: map-par slower than map, Damien Mattei, 2022/10/13
- Re: map-par slower than map, Olivier Dion, 2022/10/13
- Message not available
- Fwd: map-par slower than map, Damien Mattei, 2022/10/13
- Re: map-par slower than map, Damien Mattei, 2022/10/13
- Re: map-par slower than map, Olivier Dion, 2022/10/13
- Re: map-par slower than map, Damien Mattei, 2022/10/13
- Re: map-par slower than map, Olivier Dion, 2022/10/13