[meta-intel] [PATCH][meta-dpdk] dpdk: igb_uio: change ISR to be threaded when using with RT kernel
Liu, Yongxin
Yongxin.Liu at windriver.com
Thu Sep 19 17:12:12 PDT 2019
> -----Original Message-----
> From: Mittal, Anuj [mailto:anuj.mittal at intel.com]
> Sent: Friday, September 20, 2019 08:03
> To: Liu, Yongxin; meta-intel at yoctoproject.org
> Cc: Hao, Kexin; Yang, Liezhi
> Subject: Re: [meta-intel][PATCH][meta-dpdk] dpdk: igb_uio: change ISR to
> be threaded when using with RT kernel
>
> On Thu, 2019-09-19 at 11:29 +0800, Yongxin Liu wrote:
> > When igb_uio driver is used with RT kernel, the interrupt handler
> > should not use IRQF_NO_THREAD as a flag, since uio_event_notify
> > will call rt_spin_lock which might sleep.
> >
> > Flage "IRQF_NO_THREAD" was introduced in upstream DPDK by commit
> > "f0d1896fa
> > igb_uio: use non-threaded ISR", but this flag is incompatible with RT
> > kernel.
> > It is necessary to keep upstream code intact when using with non-RT
> > kernel.
> >
>
> Is this something that has been reported upstream?
No report from upstream.
Thanks,
Yongxin
>
> Thanks,
>
> Anuj
>
> > Signed-off-by: Yongxin Liu <yongxin.liu at windriver.com>
> > ---
> > ...nge-ISR-to-be-threaded-when-using-with-RT.patch | 84
> > +++++++++++++++++++++
> > ...nge-ISR-to-be-threaded-when-using-with-RT.patch | 85
> > ++++++++++++++++++++++
> > recipes-extended/dpdk/dpdk_17.11.6.bb | 4 +
> > recipes-extended/dpdk/dpdk_18.11.2.bb | 4 +
> > 4 files changed, 177 insertions(+)
> > create mode 100644 recipes-extended/dpdk/dpdk/dpdk-17.11-igb_uio-
> > change-ISR-to-be-threaded-when-using-with-RT.patch
> > create mode 100644 recipes-extended/dpdk/dpdk/dpdk-18.11-igb_uio-
> > change-ISR-to-be-threaded-when-using-with-RT.patch
> >
> > diff --git a/recipes-extended/dpdk/dpdk/dpdk-17.11-igb_uio-change-
> > ISR-to-be-threaded-when-using-with-RT.patch b/recipes-
> > extended/dpdk/dpdk/dpdk-17.11-igb_uio-change-ISR-to-be-threaded-when-
> > using-with-RT.patch
> > new file mode 100644
> > index 0000000..d880cd3
> > --- /dev/null
> > +++ b/recipes-extended/dpdk/dpdk/dpdk-17.11-igb_uio-change-ISR-to-be-
> > threaded-when-using-with-RT.patch
> > @@ -0,0 +1,84 @@
> > +From f78be4561de9e22ebff252feb1bdb37875d63093 Mon Sep 17 00:00:00
> > 2001
> > +From: Yongxin Liu <yongxin.liu at windriver.com>
> > +Date: Wed, 18 Sep 2019 15:23:32 +0800
> > +Subject: [PATCH] igb_uio: change ISR to be threaded when using with
> > RT kernel
> > +
> > +Non-thread ISR shouldn't call rt_spin_lock which might sleep.
> > +
> > +Here is the calltrace.
> > +
> > +[ 2034.457805] Call Trace:
> > +[ 2034.460818] <IRQ>
> > +[ 2034.463397] dump_stack+0x70/0x9a
> > +[ 2034.467281] ___might_sleep.cold+0xe1/0xf2
> > +[ 2034.471943] rt_spin_lock+0x55/0x70
> > +[ 2034.475994] ? __wake_up_common_lock+0x61/0xb0
> > +[ 2034.481004] __wake_up_common_lock+0x61/0xb0
> > +[ 2034.485842] __wake_up+0x13/0x20
> > +[ 2034.489635] uio_event_notify+0x2c/0x50 [uio]
> > +[ 2034.494560] igbuio_pci_irqhandler+0x1f/0x40 [igb_uio]
> > +[ 2034.500262] __handle_irq_event_percpu+0x5f/0x3f0
> > +[ 2034.505532] ? cpuidle_enter_state+0xd3/0x500
> > +[ 2034.510454] handle_irq_event_percpu+0x4b/0x90
> > +[ 2034.515462] handle_irq_event+0x3c/0x5b
> > +[ 2034.519864] handle_edge_irq+0xbb/0x210
> > +[ 2034.524264] handle_irq+0x23/0x30
> > +[ 2034.528143] do_IRQ+0x7e/0x150
> > +[ 2034.531764] common_interrupt+0xf/0xf
> > +[ 2034.535990] </IRQ>
> > +
> > +Upstream-Status: Inappropriate
> > +
> > +Signed-off-by: Yongxin Liu <yongxin.liu at windriver.com>
> > +---
> > + lib/librte_eal/linuxapp/igb_uio/igb_uio.c | 6 +-----
> > + 1 file changed, 1 insertion(+), 5 deletions(-)
> > +
> > +diff --git a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
> > b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
> > +index 3073bca31..3b1e1d553 100644
> > +--- a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
> > ++++ b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
> > +@@ -239,7 +239,6 @@ igbuio_pci_enable_interrupts(struct
> > rte_uio_pci_dev *udev)
> > + msix_entry.entry = 0;
> > + if (pci_enable_msix(udev->pdev, &msix_entry, 1) == 0) {
> > + dev_dbg(&udev->pdev->dev, "using MSI-X");
> > +- udev->info.irq_flags = IRQF_NO_THREAD;
> > + udev->info.irq = msix_entry.vector;
> > + udev->mode = RTE_INTR_MODE_MSIX;
> > + break;
> > +@@ -247,7 +246,6 @@ igbuio_pci_enable_interrupts(struct
> > rte_uio_pci_dev *udev)
> > + #else
> > + if (pci_alloc_irq_vectors(udev->pdev, 1, 1,
> > PCI_IRQ_MSIX) == 1) {
> > + dev_dbg(&udev->pdev->dev, "using MSI-X");
> > +- udev->info.irq_flags = IRQF_NO_THREAD;
> > + udev->info.irq = pci_irq_vector(udev->pdev, 0);
> > + udev->mode = RTE_INTR_MODE_MSIX;
> > + break;
> > +@@ -259,7 +257,6 @@ igbuio_pci_enable_interrupts(struct
> > rte_uio_pci_dev *udev)
> > + #ifndef HAVE_ALLOC_IRQ_VECTORS
> > + if (pci_enable_msi(udev->pdev) == 0) {
> > + dev_dbg(&udev->pdev->dev, "using MSI");
> > +- udev->info.irq_flags = IRQF_NO_THREAD;
> > + udev->info.irq = udev->pdev->irq;
> > + udev->mode = RTE_INTR_MODE_MSI;
> > + break;
> > +@@ -267,7 +264,6 @@ igbuio_pci_enable_interrupts(struct
> > rte_uio_pci_dev *udev)
> > + #else
> > + if (pci_alloc_irq_vectors(udev->pdev, 1, 1,
> > PCI_IRQ_MSI) == 1) {
> > + dev_dbg(&udev->pdev->dev, "using MSI");
> > +- udev->info.irq_flags = IRQF_NO_THREAD;
> > + udev->info.irq = pci_irq_vector(udev->pdev, 0);
> > + udev->mode = RTE_INTR_MODE_MSI;
> > + break;
> > +@@ -277,7 +273,7 @@ igbuio_pci_enable_interrupts(struct
> > rte_uio_pci_dev *udev)
> > + case RTE_INTR_MODE_LEGACY:
> > + if (pci_intx_mask_supported(udev->pdev)) {
> > + dev_dbg(&udev->pdev->dev, "using INTX");
> > +- udev->info.irq_flags = IRQF_SHARED |
> > IRQF_NO_THREAD;
> > ++ udev->info.irq_flags = IRQF_SHARED;
> > + udev->info.irq = udev->pdev->irq;
> > + udev->mode = RTE_INTR_MODE_LEGACY;
> > + break;
> > +--
> > +2.14.4
> > +
> > diff --git a/recipes-extended/dpdk/dpdk/dpdk-18.11-igb_uio-change-
> > ISR-to-be-threaded-when-using-with-RT.patch b/recipes-
> > extended/dpdk/dpdk/dpdk-18.11-igb_uio-change-ISR-to-be-threaded-when-
> > using-with-RT.patch
> > new file mode 100644
> > index 0000000..cc1bd79
> > --- /dev/null
> > +++ b/recipes-extended/dpdk/dpdk/dpdk-18.11-igb_uio-change-ISR-to-be-
> > threaded-when-using-with-RT.patch
> > @@ -0,0 +1,85 @@
> > +From 4f335c926ffb87749f8b1ae465445c3cf2a74128 Mon Sep 17 00:00:00
> > 2001
> > +From: Yongxin Liu <yongxin.liu at windriver.com>
> > +Date: Wed, 18 Sep 2019 17:11:09 +0800
> > +Subject: [PATCH] igb_uio: change ISR to be threaded when using with
> > RT kernel
> > +
> > +Non-thread ISR shouldn't call rt_spin_lock which might sleep.
> > +
> > +Here is the calltrace.
> > +
> > +[ 2543.830170] Call Trace:
> > +[ 2543.833179] <IRQ>
> > +[ 2543.835757] dump_stack+0x70/0x9a
> > +[ 2543.839638] ___might_sleep.cold+0xe1/0xf2
> > +[ 2543.844300] rt_spin_lock+0x55/0x70
> > +[ 2543.848353] ? __wake_up_common_lock+0x61/0xb0
> > +[ 2543.853361] __wake_up_common_lock+0x61/0xb0
> > +[ 2543.858200] __wake_up+0x13/0x20
> > +[ 2543.861993] uio_event_notify+0x2c/0x50 [uio]
> > +[ 2543.866916] igbuio_pci_irqhandler+0x1f/0x40 [igb_uio]
> > +[ 2543.872620] __handle_irq_event_percpu+0x5f/0x3f0
> > +[ 2543.877888] ? __lock_release.isra.0+0x1aa/0x310
> > +[ 2543.883071] ? debug_smp_processor_id+0x21/0x110
> > +[ 2543.888253] handle_irq_event_percpu+0x4b/0x90
> > +[ 2543.893262] handle_irq_event+0x3c/0x5b
> > +[ 2543.897663] handle_edge_irq+0xbb/0x210
> > +[ 2543.902063] handle_irq+0x23/0x30
> > +[ 2543.905944] do_IRQ+0x7e/0x150
> > +[ 2543.909564] common_interrupt+0xf/0xf
> > +[ 2543.913791] </IRQ>
> > +
> > +Upstream-Status: Inappropriate
> > +
> > +Signed-off-by: Yongxin Liu <yongxin.liu at windriver.com>
> > +---
> > + kernel/linux/igb_uio/igb_uio.c | 6 +-----
> > + 1 file changed, 1 insertion(+), 5 deletions(-)
> > +
> > +diff --git a/kernel/linux/igb_uio/igb_uio.c
> > b/kernel/linux/igb_uio/igb_uio.c
> > +index 039f5a5f6..9d7380fb7 100644
> > +--- a/kernel/linux/igb_uio/igb_uio.c
> > ++++ b/kernel/linux/igb_uio/igb_uio.c
> > +@@ -221,7 +221,6 @@ igbuio_pci_enable_interrupts(struct
> > rte_uio_pci_dev *udev)
> > + msix_entry.entry = 0;
> > + if (pci_enable_msix(udev->pdev, &msix_entry, 1) == 0) {
> > + dev_dbg(&udev->pdev->dev, "using MSI-X");
> > +- udev->info.irq_flags = IRQF_NO_THREAD;
> > + udev->info.irq = msix_entry.vector;
> > + udev->mode = RTE_INTR_MODE_MSIX;
> > + break;
> > +@@ -229,7 +228,6 @@ igbuio_pci_enable_interrupts(struct
> > rte_uio_pci_dev *udev)
> > + #else
> > + if (pci_alloc_irq_vectors(udev->pdev, 1, 1,
> > PCI_IRQ_MSIX) == 1) {
> > + dev_dbg(&udev->pdev->dev, "using MSI-X");
> > +- udev->info.irq_flags = IRQF_NO_THREAD;
> > + udev->info.irq = pci_irq_vector(udev->pdev, 0);
> > + udev->mode = RTE_INTR_MODE_MSIX;
> > + break;
> > +@@ -241,7 +239,6 @@ igbuio_pci_enable_interrupts(struct
> > rte_uio_pci_dev *udev)
> > + #ifndef HAVE_ALLOC_IRQ_VECTORS
> > + if (pci_enable_msi(udev->pdev) == 0) {
> > + dev_dbg(&udev->pdev->dev, "using MSI");
> > +- udev->info.irq_flags = IRQF_NO_THREAD;
> > + udev->info.irq = udev->pdev->irq;
> > + udev->mode = RTE_INTR_MODE_MSI;
> > + break;
> > +@@ -249,7 +246,6 @@ igbuio_pci_enable_interrupts(struct
> > rte_uio_pci_dev *udev)
> > + #else
> > + if (pci_alloc_irq_vectors(udev->pdev, 1, 1,
> > PCI_IRQ_MSI) == 1) {
> > + dev_dbg(&udev->pdev->dev, "using MSI");
> > +- udev->info.irq_flags = IRQF_NO_THREAD;
> > + udev->info.irq = pci_irq_vector(udev->pdev, 0);
> > + udev->mode = RTE_INTR_MODE_MSI;
> > + break;
> > +@@ -259,7 +255,7 @@ igbuio_pci_enable_interrupts(struct
> > rte_uio_pci_dev *udev)
> > + case RTE_INTR_MODE_LEGACY:
> > + if (pci_intx_mask_supported(udev->pdev)) {
> > + dev_dbg(&udev->pdev->dev, "using INTX");
> > +- udev->info.irq_flags = IRQF_SHARED |
> > IRQF_NO_THREAD;
> > ++ udev->info.irq_flags = IRQF_SHARED;
> > + udev->info.irq = udev->pdev->irq;
> > + udev->mode = RTE_INTR_MODE_LEGACY;
> > + break;
> > +--
> > +2.14.4
> > +
> > diff --git a/recipes-extended/dpdk/dpdk_17.11.6.bb b/recipes-
> > extended/dpdk/dpdk_17.11.6.bb
> > index 40edd8b..b9ce69c 100644
> > --- a/recipes-extended/dpdk/dpdk_17.11.6.bb
> > +++ b/recipes-extended/dpdk/dpdk_17.11.6.bb
> > @@ -13,6 +13,10 @@ SRC_URI += "\
> > file://dpdk-17.11-mk-disable-warning-for-packed-member-
> > pointer.patch \
> > file://dpdk-17.11-igb_uio-fix-build-on-Linux-5.3-for-fall-
> > through.patch \
> > "
> > +SRC_URI += "${@bb.utils.contains('PREFERRED_PROVIDER_virtual/kernel'
> > ,\
> > + 'linux-yocto-rt',\
> > + 'file://dpdk-17.11-igb_uio-change-ISR-to-be-threaded-
> > when-using-with-RT.patch',\
> > + '',d)}"
> >
> > do_install_append () {
> > # Remove the unneeded dir
> > diff --git a/recipes-extended/dpdk/dpdk_18.11.2.bb b/recipes-
> > extended/dpdk/dpdk_18.11.2.bb
> > index 5291bdc..1d2ebb6 100644
> > --- a/recipes-extended/dpdk/dpdk_18.11.2.bb
> > +++ b/recipes-extended/dpdk/dpdk_18.11.2.bb
> > @@ -12,6 +12,10 @@ LIC_FILES_CHKSUM = "
> > file://license/gpl-2.0.txt;md5=b234ee4d69f5fce4486a80fdaf4a4
> > SRC_URI += "\
> > file://dpdk-18.11-igb_uio-fix-build-on-Linux-5.3-for-fall-
> > through.patch \
> > "
> > +SRC_URI += "${@bb.utils.contains('PREFERRED_PROVIDER_virtual/kernel'
> > ,\
> > + 'linux-yocto-rt',\
> > + 'file://dpdk-18.11-igb_uio-change-ISR-to-be-threaded-
> > when-using-with-RT.patch',\
> > + '',d)}"
> >
> > do_install_append () {
> > # Remove the unneeded dir
More information about the meta-intel
mailing list