Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline2.0 step 7: Make all the code use API to access comp_buffer lists #9802

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

marcinszkudlinski
Copy link
Contributor

This PR makes the pipeline management code use API to access comp_buffer
it had been done similarly in the modules some time ago

Since now all access to comp_buffer lists should be performed through an API
There's one exception - unit testing that perform an unusual operations, that should not be available in other cases

module_source_status_count in fact counts
"state" field of all components.
Name changed to module_source_state_count

Signed-off-by: Marcin Szkudlinski <marcin.szkudlinski@intel.com>
@marcinszkudlinski marcinszkudlinski changed the title Make all the code use API to access comp_buffer lists Pipeline2.0 step 7: Make all the code use API to access comp_buffer lists Jan 29, 2025
src/audio/module_adapter/module_adapter.c Outdated Show resolved Hide resolved
src/audio/module_adapter/module_adapter.c Outdated Show resolved Hide resolved
@@ -309,9 +309,6 @@ int ipc_comp_free(struct ipc *ipc, uint32_t comp_id)
struct comp_buffer *buffer = container_of(clist, struct comp_buffer, sink_list);

buffer->sink = NULL;
/* Also if it isn't shared - we are about to modify uncached data */
dcache_writeback_invalidate_region(uncache_to_cache(buffer),
sizeof(*buffer));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment about modifying uncached data is a left-over from the time, when we only accessed buffer list headers uncached, even when the buffer object itself was cached. We don't do that any more - cached buffer objects are added to the lists via cached pointers and uncached ones - via uncached. So, yes, this cache synchronisation doesn't seem to be needed any more.

src/ipc/ipc4/helper.c Outdated Show resolved Hide resolved
list_init(&buffer->source_list);
list_init(&buffer->sink_list);
comp_buffer_reset_source_list(buffer);
comp_buffer_reset_sink_list(buffer);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

each change in this commit is simple, but there are so many of them... Maybe this could be split in 1 commit per accessor?

Copy link
Collaborator

@kv2019i kv2019i left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last commit is hard to review, but looks fine. Guennadi spotted some issues, especially the attach/detach needs address, with those this seems good to go.

comp_buffer::sink_list field used to have 2 meanings:

 - a list connector of all data receivers from the module
   in this case list is based on
       comp_dev::bsource_list

 - a list connector for data receivers for extra modules
   allocated when raw data processing is in use
   in this case list is based on
       processing_module::sink_buffer_list

This commit separate both cases
case 1 - no changes
case 2 -
 list is now based on processing_module::raw_data_buffers_list
 list is connected by a new field comp_buffer::buffers_list

Signed-off-by: Marcin Szkudlinski <marcin.szkudlinski@intel.com>
All control structures of buffers shared between cores
are now stored in a non-cached memory alias.
Cache writeback_invalidate is no longer needed here

Signed-off-by: Marcin Szkudlinski <marcin.szkudlinski@intel.com>
All componetns had been modified to use buffer's API
some time ago, those 2 were either missed or have been
modified later

Signed-off-by: Marcin Szkudlinski <marcin.szkudlinski@intel.com>
this commit adds 4 methods to comp_buffer API and refactors
one macro to a static inline function in .c file
the reason - macro was used once only and provides
no type control

Signed-off-by: Marcin Szkudlinski <marcin.szkudlinski@intel.com>
@marcinszkudlinski marcinszkudlinski force-pushed the pipeline2_0_7 branch 2 times, most recently from 91a562f to 4b5fc69 Compare February 4, 2025 16:13
Copy link
Member

@lgirdwood lgirdwood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but can we add some more comments/context regarding "raw" processing mode.We can merge after that.

@@ -778,7 +778,7 @@ cadence_codec_process(struct processing_module *mod,
}

/* do not proceed with processing if not enough free space left in the local buffer */
local_buff = list_first_item(&mod->sink_buffer_list, struct comp_buffer, sink_list);
local_buff = list_first_item(&mod->raw_data_buffers_list, struct comp_buffer, buffers_list);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do we mean by "raw" data processing here, how is this different from source/sink ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"raw data" is used at many places

/*
 * helpers to determine processing type
 * Needed till all the modules use PROCESSING_MODE_SINK_SOURCE
 */
#define IS_PROCESSING_MODE_RAW_DATA(mod) ((mod)->proc_type == MODULE_PROCESS_TYPE_RAW)

	/**
	 * process_raw_data (depreciated)
	 *	- sources are input_stream_buffer[]
	 *	    - sources[].data is a pointer to raw audio data
	 *	- sinks are output_stream_buffer[]
	 *	    - sinks[].data is a pointer to raw audio data
	 */
	int (*process_raw_data)(struct processing_module *mod,
				struct input_stream_buffer *input_buffers,
				int num_input_buffers,
				struct output_stream_buffer *output_buffers,
				int num_output_buffers);

@lgirdwood
Copy link
Member

lgirdwood commented Feb 11, 2025

@marcinszkudlinski can you check CI testbench, looks like we maybe reading next if it does not exist ?

----------------------------------------------------------
==8685== Memcheck, a memory error detector
==8685== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==8685== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==8685== Command: ../../testbench/build_testbench/install/bin/sof-testbench4 -r 48000 -R 48000 -c 2 -n 2 -b S[16](https://github.com/thesofproject/sof/actions/runs/13139850217/job/36664107366?pr=9802#step:7:17)_LE -p 1,2 -t ../../build_tools/topology/topology2/development/sof-hda-benchmark-gain16.tplg -i chirp_test_in_861801.raw -o chirp_test_out_861801.raw
==8685== 
trace: [[17](https://github.com/thesofproject/sof/actions/runs/13139850217/job/36664107366?pr=9802#step:7:18)38685764.730189] <inf> (ll_schedule.c:114) ll_scheduler_init()
trace: [1738685764.742810] <inf> (edf_schedule.c:112) edf_scheduler_init()
Info: Found control type 1: Analog Playback GAIN volume
Info: Found control type 1: Analog Capture GAIN volume
warning: failed  assigning pipeline for iDisp3 Tx
warning: failed  assigning pipeline for codec2_in
warning: failed  assigning pipeline for Alt Analog CPU Playback
warning: failed  assigning pipeline for codec2_out
warning: failed  assigning pipeline for Analog CPU Capture
warning: failed  assigning pipeline for iDisp1_out
warning: failed  assigning pipeline for Digital CPU Capture
warning: failed  assigning pipeline for iDisp2_out
warning: failed  assigning pipeline for Alt Analog CPU Capture
warning: failed  assigning pipeline for iDisp3_out
trace: [1738685764.789416] <inf> (handler.c:1562) rx	: 0x11000004|0
trace: [1738685764.791740] <inf> (pipeline-graph.c:116) pipeline new pipe_id 0 priority 0
trace: [1738685764.795939] <inf> (handler.c:1562) rx	: 0x4000009a|0x16
trace: [1738685764.803076] <inf> (handler.c:1562) rx	: 0x40000006|0x14
trace: [1738685764.805748] <inf> (handler.c:1562) rx	: 0x4500009a|0x6
trace: [1738685764.807522] <inf> (ipc-helper.c:54) buffer new size 0x300 id 0.0 flags 0x0
trace: [1738685764.811637] <inf> (pipeline-graph.c:[18](https://github.com/thesofproject/sof/actions/runs/13139850217/job/36664107366?pr=9802#step:7:19)2) connect buffer 0 as sink
trace: [1738685764.812363] <inf> (pipeline-graph.c:184) connect buffer 0 as source
trace: [1738685764.814630] <inf> (handler.c:1562) rx	: 0x11010002|0
trace: [1738685764.814680] <inf> (pipeline-graph.c:116) pipeline new pipe_id 1 priority 0
trace: [1738685764.814825] <inf> (handler.c:1562) rx	: 0x4000009c|0x10016
trace: [1738685764.815830] <inf> (handler.c:1562) rx	: 0x45000006|0x9c
trace: [1738685764.815883] <inf> (ipc-helper.c:54) buffer new size 0x300 id 0.0 flags 0x0
trace: [1738685764.816142] <inf> (pipeline-graph.c:182) connect buffer 0 as sink
trace: [1738685764.816184] <inf> (pipeline-graph.c:184) connect buffer 0 as source
trace: [1738685764.817720] <inf> (handler.c:1562) rx	: 0x13000003|0
trace: [1738685764.820690] <wrn> (ipc-helper.c:238) ipc_pipeline_complete(): no scheduling component specified, use comp 0x6
trace: [1738685764.824927] <inf> (handler.c:1562) rx	: 0x13010003|0
trace: [1738685764.825255] <wrn> (ipc-helper.c:238) ipc_pipeline_complete(): no scheduling component specified, use comp 0x9c
trace: [1738685764.825702] <inf> (handler.c:1562) rx	: 0x13000004|0
trace: [1738685764.842878] <inf> (pipeline-stream.c:387) pipe trigger cmd 7
trace: [1738685764.846125] <inf> (handler.c:1562) rx	: 0x13010004|0
trace: [1738685764.846428] <inf> (pipeline-stream.c:387) pipe trigger cmd 7
trace: [1738685764.854095] <err> (handler.c:523) unexpected delayed reply
trace: [1738685764.858744] <err> (handler.c:523) unexpected delayed reply
trace: [1738685765.153775] <inf> (handler.c:1562) rx	: 0x13000003|0
trace: [1738685765.154114] <inf> (pipeline-stream.c:387) pipe trigger cmd 2
trace: [1738685765.155054] <inf> (handler.c:1562) rx	: 0x13010003|0
trace: [1738685765.155105] <inf> (pipeline-stream.c:387) pipe trigger cmd 2
trace: [1738685765.155873] <err> (handler.c:523) unexpected delayed reply
trace: [1738685765.156017] <err> (handler.c:523) unexpected delayed reply
trace: [1738685765.156280] <inf> (handler.c:1562) rx	: 0x13000002|0
trace: [1738685765.156770] <inf> (pipeline-stream.c:387) pipe trigger cmd 0
trace: [1738685765.160522] <inf> (handler.c:1562) rx	: 0x13010002|0
trace: [1738685765.160578] <inf> (pipeline-stream.c:387) pipe trigger cmd 0
trace: [1738685765.176630] <inf> (handler.c:1562) rx	: 0x46000006|0x9c
trace: [1738685765.180692] <inf> (handler.c:1562) rx	: 0x12000000|0
==8685== Invalid read of size 8
==8685==    at 0x111569: comp_dev_get_next_data_producer (component.h:656)
==8685==    by 0x111569: ipc_pipeline_module_free (helper.c:312)
==8685==    by 0x111569: ipc_pipeline_free.cold (helper.c:349)
==8685==    by 0x11C0C6: ipc4_delete_pipeline (handler.c:149)
==8685==    by 0x11C0C6: ipc4_process_glb_message (handler.c:824)
==8685==    by 0x11C0C6: ipc_cmd (handler.c:1576)
==8685==    by 0x116D04: tb_ipc_message (topology_ipc4.c:50)
==8685==    by 0x116D04: tb_mq_cmd_tx_rx.constprop.0 (topology_ipc4.c:69)
==8685==    by 0x1[19](https://github.com/thesofproject/sof/actions/runs/13139850217/job/36664107366?pr=9802#step:7:20)3B1: tb_delete_pipeline (topology_ipc4.c:1596)
==8685==    by 0x115BE5: tb_free_pipelines.isra.0 (utils_ipc4.c:565)
==8685==    by 0x1162FD: tb_free_all_pipelines (utils_ipc4.c:578)
==8685==    by 0x1128D5: pipline_test (testbench.c:412)
==8685==    by 0x1128D5: main (testbench.c:490)
==8685==  Address 0x110 is not stack'd, malloc'd or (recently) free'd

add comp_dev_for_each_consumer_safe and comp_dev_for_each_producer_safe
Those macros allow iteration through all buffers allowing
buffers deletion

Signed-off-by: Marcin Szkudlinski <marcin.szkudlinski@intel.com>
Unit test should also - if possible - use buffer API
There are, however, some parts of testing that still
use direct access to comp_buffer internals.

Signed-off-by: Marcin Szkudlinski <marcin.szkudlinski@intel.com>
This commit make all pipeline management code
using comp_buffer API. Since now, no access to comp_buffer
internals should be performed by the SOF code
(with exception of uint test)

Signed-off-by: Marcin Szkudlinski <marcin.szkudlinski@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants