python - Is it possible to use complex mapping functions with Polars? - Stack Overflow

admin管理员组

文章数量:1122846

I need to apply this mapper function to a Series of user ids:

def generate_lightfm_recs_mapper(
        model: object,
        item_ids: list,
        known_items: dict,
        user_features: list,
        item_features: list,
        N: int,
        user_mapping: dict,
        item_inv_mapping: dict,
        num_threads: int = 4
        ):
    def _recs_mapper(user):
        user_id = user_mapping[user]
        recs = model.predict(
            user_id,
            item_ids,
            user_features = user_features,
            item_features = item_features,
            num_threads = num_threads)
        
        additional_N = len(known_items[user_id]) if user_id in known_items else 0
        total_N = N + additional_N
        top_cols = np.argpartition(recs, -np.arange(total_N))[-total_N:][::-1] 
        
        final_recs = [item_inv_mapping[item] for item in top_cols]
        if additional_N > 0:
            filter_items = known_items[user_id]
            final_recs = [item for item in final_recs if item not in filter_items]
        return final_recs[:N]
    return _recs_mapper

This is the object:

# init mapper to get predictions
mapper = generate_lightfm_recs_mapper(
    lfm_model, 
    item_ids = all_cols, 
    known_items = dict(),
    N = top_N,
    user_features = None, 
    item_features = None, 
    user_mapping = lightfm_mapping['users_mapping'],
    item_inv_mapping = lightfm_mapping['items_inv_mapping'],
    num_threads = 20
)

In Pandas I would do it just like this:

local_test_preds['item_id'] = local_test_preds['user_id'].map(mapper)

But there's so much data I'm afraid it would take extremely long time to execute.

Is it possible to rewrite this in Polars? I searched for map_elements and map_batches functions but in all examples there aren't as many input arguments and they are initialized inside .with_columns method. However, I do my initialization earlier.

Thanks in advance.

本文标签： pythonIs it possible to use complex mapping functions with PolarsStack Overflow