Section topics
==============

.. automodule:: section_topics.pipeline

.. function:: parse(wikitext, section_level=SECTION_LEVEL, section_zero_heading=SECTION_ZERO_HEADING)

    .. note::

        This is the core function responsible for the data heavy lifting.
        It's implemented as a PySpark user-defined function (:func:`pyspark.sql.functions.udf`).
        It must be called by the :func:`parse_wikitext` currying function,
        which seems the only way to pass an optional denylist of section headings.


    Parse a wikitext string into a list of dicts.
    Each dict represents a section within the given wikitext and has the following keys:

    - ``index`` - section number, ``0`` for the leading section
    - ``heading`` - section heading, ``section_zero_heading``'s value for the leading section
    - ``links`` - list of section wikilinks
    - ``images`` - list of section image file names

    A section heading is normalized through :func:`wikitext_headings_to_anchors`.
    Clean wikilinks with mwparserfromhell's
    `strip_code() <https://mwparserfromhell.readthedocs.io/en/v0.6.4/api/mwparserfromhell.html#mwparserfromhell.wikicode.Wikicode.strip_code>`_.
    This can lead to empty strings that should be filtered.

    :param wikitext: a wikitext
    :type wikitext: str
    :param section_level: (optional) a section hierarchy level to be extracted
    :type section_level: int
    :param section_zero_heading: (optional) a heading reserved to the lead section
    :type section_zero_heading: str
    :return: the list of *(index, heading, links, images)* section dictionaries
    :rtype: List[str]