Re: Guix Survey (follow up on "How can we decrease the cognitive overhead for contributors?")

[Peter Pronai]

Wilko Meyer <w@wmeyer.eu> writes:

> Hi Guix,
>
> I haven't had enough time to read up on every topic that has been
> mentioned in the "How can we decrease the cognitive overhead for
> contributors?" discussion as at some point it got quite a lot to
> follow. At one point[0] there was a discussion on having a survey to get
> a better picture on and quantify what potential blockers are to engage
> with/contribute to Guix; which seems, if done right (as surveys have to
> be carefully crafted), a good idea; especially with the prospect of
> repeating it annually as a means to check if issues got
> better/priorities in Guixes userbase change and so on. If there's a
> consensus on doing this, I'd be happy to contribute some of my time to
> get things going (would creating a issue on guixes bug tracker for this
> topic be a good idea? how are these non-code contrib. topics handled?).
>
> Before writing this mail, I had a look on how other projects handle
> these kind of surveys, in particular:
>
> - the emacs user survey[1]
> - the nix community survey[2]
> - the curl user survey[3]
> - the fennel survey[4]
>
> I identified a few key themes that could be useful for a guix user
> survey as well. I plan on doing a more extensive summary on this later
> this weekend if my time allows it, for now a loose collection of
> ideas/list of what, in my subjective opinion, stood out and what most
> surveys had in common should do to hopefully get a discussion on this
> started:
>
> - the emacs user survey specifically asked for elisp profiency; mapping
>   out the Guile profiency of guixes community could be feasible.
> - fennel as well as emacs had questions on which programming languages
>   their community uses; in the regards on recent discussions on
>   guix-devel on developer ecosystems[4] this could help to identify if
>   there are any shortcomings in providing importers/packages for certain
>   languages that may be used by guix users.
> - the nix survey specifically asked for the environments and context nix
>   is being used in; it'd be interesting to see where and for what
>   purpose people are using Guix.
> - most surveys had, some more some less extensive, demographic
>   questions and questions mapping out how many years people have been
>   programming.
>
> Specifially in the lights of the original discussion/regarding
> contributions:
>
> - I think that the "Where do you discuss Fennel or interact with other
>   Fennel developers" question of fennels survey should be asked for guix
>   as well, to get a grasp on which platforms are being used to discuss
>   all things guix.
> - the curl user survey[6] did a pretty good job in mapping out what
>   prevents users from contributing (p.20) as well as mapping out what
>   areas of the project are regarding as good/which have room for
>   improvements (p.24-26)
> - fennel asked for "the biggest problems you have using Fennel", it had
>   a "If you haven't hacked on Fennel itself, why not?" question as
>   well. I personally think this could be good to assess potential pain
>   points/blockers for Guix as well. Fennel also asked for "favorite
>   features" which could be a nice way to map out which parts of Guix are
>   popular.
>
> Last, the nix user survey allowed free-form responses. Having a
> qualitative research component to a survey could help getting better
> results (especially when identifying problems in using guix/blockers in
> contributing and so on); but evaluating these is pretty time extensive
> and dependant on how much resources people have to compile a list of
> findings/results from a prospective survey.
>
> What could the next steps be to get this going?
>
> [0]: https://lists.gnu.org/archive/html/guix-devel/2023-09/msg00086.html
> [1]: https://emacssurvey.org/
> [2]: https://discourse.nixos.org/t/2022-nix-survey-results/18983
> [3]: https://daniel.haxx.se/blog/2022/06/16/curl-user-survey-2022-analysis/
> [4]: https://fennel-lang.org/survey/2022
> [5]: https://lists.gnu.org/archive/html/guix-devel/2023-07/msg00152.html
> [6]: https://daniel.haxx.se/media/curl-user-survey-2023-analysis.pdf

I definitely vote for having a free form field too, and also an extra
one for feedback on the survey.  It might not be easy to turn it into
quantitative data, but if a lot of people mention certain key words,
that should be both easy to grep for and very apparent even for a casual
reader.