pandas - Snowflake ML SimpleImputer does not work as expected - Stack Overflow

admin管理员组

文章数量:1122832

I am trying to predict in a python worksheet with my custom xgboost model which I deployed to Snowflake Model Registry. I am reading my table with modin pandas read_snowflake, than I am trying to use SimpleImputer but keep getting:

raise NotImplementedError(message) NotImplementedError: Snowpark pandas does not support the DataFrame interchange protocol method __dataframe__. To use Snowpark pandas DataFrames with third-party libraries that try to call the __dataframe__ method, please convert this Snowpark pandas DataFrame to pandas with to_pandas()

I understand it wants me to convert to native pandas but since it has limits on snowflake I do not want that since speed is cost important for this case. I do not understand why snowpark own method does not work?

Here is my code:

import snowflake.snowpark as snowpark
from snowflake.snowpark.functions import col
from snowflake.ml.registry import Registry
import modin.pandas as pd
import snowflake.snowpark.modin.plugin
from snowflake.ml.modeling.impute import SimpleImputer

def main(session: snowpark.Session): 
 model_registry = Registry(session=session, database_name="MY_DB", schema_name="PUBLIC")
 model = model_registry.get_model('MY_MODEL').version('v1')
 df=pd.read_snowflake('MY_TABLE')


 numerical_features = [ MY_NUMERIC_COLUMNS]
 categorical_features = [MY_CAT_COLUMNS]

 for col in numerical_features:
     df[col] = pd.to_numeric(df[col], errors='coerce').astype(float)

 
 X_custom_test = df.drop(columns=[""])
 X_custom_test = X_custom_test.applymap(lambda x: np.nan if pd.isna(x) else x)
 
 imputer_num = SimpleImputer(strategy="mean", input_cols=numerical_features)
 imputer_num.fit(X_custom_test)
 X_custom_test = imputer_num.transform(X_custom_test)

 imputer_cat = SimpleImputer(strategy="most_frequent", input_cols=categorical_features)
 imputer_cat.fit(X_custom_test)
 X_custom_test = imputer_cat.transform(X_custom_test)
 
 #test_y_pred = model.run(X_custom_test, function_name="predict")

本文标签： pandasSnowflake ML SimpleImputer does not work as expectedStack Overflow