PERF: avoid copy in _obj_with_exclusions, _selected_obj

[jbrockmendel]

• closes #xxxx (Replace xxxx with the GitHub issue number)
• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

cc @rhshadrach

import pandas as pd
import numpy as np
import psutil
proc = psutil.Process()

np.random.seed(456123)
df = pd.DataFrame(
    np.random.randn(10_000_000, 5),
    columns=["A", "B", "C", "D", "E"],
)

grps = np.arange(10).repeat(1_000_000)
gb = df.groupby(grps)

gb2 = gb[["C"]]

In [3]: np.shares_memory(gb2._obj_with_exclusions["C"], df["C"])
True   # <- PR
False  # <- main

In [5]: proc.memory_info()
Out[5]: pmem(rss=658894848, vms=36076212224, pfaults=169250, pageins=0)  <- main
Out[5]: pmem(rss=578924544, vms=35993063424, pfaults=149692, pageins=0)  <- PR

[rhshadrach]

lgtm; I think we can remove a case the logic in _selected_obj, but might be wrong.

[rhshadrach]

I think any time self.obj[self._selection] results in a Series will be when self is a SeriesGroupBy and self.obj is a Series, which is handled above.

If the frame has duplicate columns, then we can windup in the case where self._selection is hashable and self is a DataFrameGroupBy, but in this case I think self.obj[self_selection] will then agree with self._obj_with_exclusions (both frames).

So I think we can still return obj_with_exclusions here.

[jbrockmendel]

That logic works for SeriesGroupBy/DataFrameGroupBy, but this code path is shared by Resampler, which defines _gotitem kludgily. Until we change that, test_resample_groupby_agg_listlike would fail if we return obj_with_exclusions here

[jbrockmendel]

Until we change that

(this is part of why i pinged you about the non-unique columns in agg_dict_like. If we can make sure to always pass subset to _gotitem, i think that makes it a lot easier to de-kludge Resample._gotitem)

[rhshadrach]

Do you think the memory perf improvement is worth a note in the whatsnew?

[jbrockmendel]

Do you think the memory perf improvement is worth a note in the whatsnew?

added

[rhshadrach]

Thanks @jbrockmendel

[jbrockmendel]

i think the describe PR will need rebasing (might want to wait until after #51113 is merged) and it should be good to go