[yocto] Definition of Yocto tasks

Thu May 8 22:47:44 PDT 2014

Bob, 

This is good information regarding the process you had to go through and what would have helped along the way.  I am currently working on a new chapter that documents the do_<task> tasks in the YP Reference Manual.  

Scott

>-----Original Message-----
>From: yocto-bounces at yoctoproject.org [mailto:yocto-
>bounces at yoctoproject.org] On Behalf Of Bob Feretich
>Sent: Thursday, May 08, 2014 4:25 PM
>To: yocto at yoctoproject.org
>Subject: Re: [yocto] Definition of Yocto tasks
>
>
>On 5/7/2014 5:58 AM, Paul Eggleton wrote:
>> On Tuesday 06 May 2014 15:23:45 Bob Feretich wrote:
>>> On 5/6/2014 2:38 AM, Paul Eggleton wrote:
>>>> On Tuesday 06 May 2014 06:47:08 Rifenbark, Scott M wrote:
>>>>>> -----Original Message-----
>>>>>> From: yocto-bounces at yoctoproject.org [mailto:yocto-
>>>>>> bounces at yoctoproject.org] On Behalf Of Bob Feretich
>>>>>> Sent: Monday, May 05, 2014 4:52 PM
>>>>>> To: yocto at yoctoproject.org
>>>>>> Subject: [yocto] Definition of Yocto tasks
>>>>>>
>>>>>> Is there a document that provides a detailed definition of what
>>>>>> each these tasks do?
>>>>>> Yocto, OE, and bitbake manuals tell us how to command a specific
>>>>>> task to be run, but not what they do.
>>>>>>
>>>>>> The task name provides a good hint sometimes, but often that is
>>>>>> not enough.
>>>>>> Users shouldn't have to read the python code for this information.
>>>>>> These descriptions should be a part of the yocto manual.
>>>>>>
>>>>>> Sections 5.3.4 to 5.3.11 of the Yocto mega-manual provide a good
>>>>>> overview of some of these tasks in the context of a workflow, but
>>>>>> doesn't mention most of these tasks and doesn't go into enough
>>>>>> detail on the tasks it discusses.
>>>>>>
>>>>>> It wouldn't be appropriate to add more detail at that point in the
>>>>>> manual, but it would be appropriate to include details regarding
>>>>>> all of the tasks in an appendix.
>>>>>>
>>>>>> Examples...
>>>>>> The manual states that do_fetch fetches source, but doesn't state
>>>>>> that when git is used whether it performs a pull, fetch, or clone.
>>>>>> What are the common failure conditions of do_fetch? And what
>>>>>> should the user do to fix the problem?
>>>>>>
>>>>>> do_build is mentioned a few times, but there is no reference to it
>>>>>> being the default task or what tasks are invoked by do_build and
>>>>>> which are omitted.
>>>>>>
>>>>>> do_rm_work and do_wm_work_all are not mentioned at all.
>>>>>>
>>>>>> Result of listtasks:
>>>>>> do_fetchall
>>>>>> ... snipped ...
>>>>>> do_rm_work_all
>>>>>>
>>>>>> I don't have the knowledge to create such an appendix, but I
>>>>>> volunteer to be a proofreader.
>>>>> An appendix for a reference of these tasks seems like a good idea.
>>>> FYI, you may already have seen it but we have a bit of coverage for
>>>> the
>>>> common>
>>>> tasks in the following section of the manual:
>>>>     http://www.yoctoproject.org/docs/current/ref-manual/ref-
>manual.html#clo
>>>>     ser-look
>>> This section seems to be a more polished version of the mega manual
>>> Sections 5.3.4 to 5.3.11.
>> They should be identical. The mega manual is simply the other manuals
>> combined together...
>>
>>> It provides an overview to the build process, but its still at a very
>>> high level (few details).
>> Could you expand on the details are you looking for that you're not
>> finding there?
>I was looking for more detailed information because I was trying to trouble
>shoot failures that occurred during the "do_fetch" of some source.
>The most common failure was the inability to access any of the servers that
>contained the source. This was probably due to those servers being
>temporarily down for maintenance. Simply restarting the bitbake fixed those
>problems.
>(A common failure that users should be told not to be concerned about. I did
>read that somewhere, but I don't remember where.)
>
>The next problem was a recipe error effecting "do_install" of some recipe. I
>troubleshot this and got a fix from the Angstrom people. while
>troubleshooting (doing lots of double Control-Cs), I believe that I hung the
>"do_fetch" mechanism. (some "do_fetch" tasks were being executed in
>parallel with the broken "do_install"). After the "do_install" problem was
>fixed, "do_fetch" seemed to run, but didn't transfer any data (no network
>traffic), and it would eventually time out.
>
>That raised the question regarding the use of lock files. (Did my Control-Cs
>leave a Yocto lock file intact that needed to be cleaned up?
>Did "do_fetch" use any locking mechanism? How could I do a partial "clean"
>that would fix the problem without setting my progress back further than
>necessary?)
>
>During my search for info, I found e-mail discussions of  clean, cleanstate, and
>cleanall.  My "do_fetch" failure was occurring at about step 6900 of 8100 build
>steps. With my internet connection, restarting would have set me back 48 hrs,
>with no guarantee that the restart would result in a fix. I was looking for ways
>to "clean" the condition without having to completely start over. Eventually, I
>gave up and did the cleanall... it did not fix the problem.
>Even though none of cleanxxx tasks worked for me in fixing the "do_fetch"
>hang, info about them should have been easier to find.
>
>I fixed the problem with a shotgun approach.
>I erased the entire Yocto build directory, rebooted my host build system, and
>power cycled my network router and DSL modem. This worked, but was
>probably more than I needed to do. (I now think that my "do_fetch" hang was
>do to not properly reinitializing a firewall port.)
>
>My searches also led me to the "do_fetchall" task, as I has just visited with a
>friend whose network connection was 10x better than mine. Had I know
>about fetchall, I would have used it during the visit.
>
>In general, the appendix should contain a description of the program logic of
>the task, the task's intended use, and a discussion of common errors (and
>fixes) that can occur during execution of the task.
>
>The most probable readers of the appendix would be:
>* someone trying to troubleshoot a problem that is occuring during the task.
>* someone who saw an e-mail reference for a use for the task and wants to
>understand it better.
>
>Regards,
>Bob
>>
>>>> If we wanted to add an appendix to list them all (and it might be
>>>> worth us doing so) a good starting point would be the task
>>>> descriptions in
>>>>
>>>> documentation.conf:
>>>>     http://cgit.openembedded.org/openembedded-
>core/tree/meta/conf/documenta
>>>>     tion.conf
>>> This file at least provides one sentence on most tasks. (do_setscene
>>> is missing, maybe more).
>> do_setscene itself isn't a task that we have. Setscene equivalents
>> exist for all of the sstate-enabled tasks that we have i.e.
>> do_populate_sysroot_setscene is the setscene equivalent of
>> do_populate_sysroot. We should touch on it elsewhere as well, but FYI
>> we do have an explanation of the setscene process in the BitBake manual:
>>
>> http://www.yoctoproject.org/docs/1.6/bitbake-user-manual/bitbake-user-
>> manual.html#setscene
>>
>> Cheers,
>> Paul
>>
>
>--
>_______________________________________________
>yocto mailing list
>yocto at yoctoproject.org
>https://lists.yoctoproject.org/listinfo/yocto