Coverage for ibllib/pipes/__init__.py: 33%

1"""IBL preprocessing pipeline. 

2 

3This module concerns the data extraction and preprocessing for IBL data. The lab servers routinely 

4call :func:`local_server.job_creator` to search for new sessions to extract. The job creator 

5registers the new session to Alyx (i.e. creates a new session record on the database), if required, 

6then deduces a set of tasks (a.k.a. the pipeline[*]_) from the 'experiment.description' file at the 

7root of the session (see :func:`dynamic_pipeline.make_pipeline`). If no file exists one is created, 

8inferring the acquisition hardware from the task protocol. The new session's pipeline tasks are 

9then registered for another process (or server) to query. 

10 

11Another process calls :func:`local_server.task_queue` to get a list of queued tasks from Alyx, then 

12:func:`local_server.tasks_runner` to loop through tasks. Each task is run by calling 

13:func:`tasks.run_alyx_task` with a dictionary of task information, including the Task class and its 

14parameters. 

15 

16.. [*] A pipeline is a collection of tasks that depend on one another. A pipeline consists of 

17 tasks associated with the same session path. Unlike pipelines, tasks are represented in Alyx. 

18 A pipeline can be recreated given a list of task dictionaries. The order is defined by the 

19 'parents' field of each task. 

20 

21Notes 

22----- 

23All new tasks are subclasses of the base_tasks.DynamicTask class. All others are defunct and shall 

24be removed in the future. 

25""" 

26 

27 

28def assign_task(task_deck, session_path, task, **kwargs): (empty)

29 """ 

30 Assigns a task to a task deck with the task name as key. 

31 

32 This is a convenience function when creating a large task deck. 

33 

34 Parameters 

35 ---------- 

36 task_deck : dict 

37 A dictionary of tasks to add to. 

38 session_path : str, pathlib.Path 

39 A session path to pass to the task. 

40 task : ibllib.pipes.tasks.Task 

41 A task class to instantiate and assign. 

42 **kwargs 

43 Optional keyword arguments to pass to the task. 

44 

45 Examples 

46 -------- 

47 >>> from ibllib.pipes.video_tasks import VideoCompress 

48 >>> task_deck = {} 

49 >>> session_path = './subject/2023-01-01/001' 

50 >>> assign_task(task_deck, session_path, VideoCompress, cameras=('left',)) 

51 {'VideoCompress': <ibllib.pipes.video_tasks.VideoCompress object at 0x0000020461E762D0>} 

52 

53 Using partial for convenience 

54 

55 >>> from functools import partial 

56 >>> assign = partial(assign_task, task_deck, session_path) 

57 >>> assign(VideoCompress, cameras=('left',)) 

58 """ 

59 t = task(session_path, **kwargs) 

60 task_deck[t.name] = t