Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doris无法加载表结构数据 #2344

Open
janfengye opened this issue Aug 26, 2024 · 2 comments
Open

doris无法加载表结构数据 #2344

janfengye opened this issue Aug 26, 2024 · 2 comments
Assignees

Comments

@janfengye
Copy link

测试链接是成功的,也能成功保存,就是不能同步表数据
2024-08-26 21:12:23.037 [http-nio-0.0.0.0-2024-exec-11] INFO datart.data.provider.JdbcDataProvider : jdbc source '5ad4570508f544b8930156b8ef95339d-Doris' updated, source has been reset
2024-08-26 21:12:23.043 [pool-2-thread-2] INFO d.data.provider.jdbc.DataSourceFactoryDruidImpl : druid data source created (DataSource-1111789274)
2024-08-26 21:12:23.049 [pool-2-thread-2] INFO com.alibaba.druid.pool.DruidDataSource : {dataSource-6} inited
2024-08-26 21:12:23.063 [http-nio-0.0.0.0-2024-exec-11] WARN com.alibaba.druid.pool.DruidAbstractDataSource : discard long time none received connection. , jdbcUrl : jdbc:mysql://localhost:3333/datart?&allowMultiQueries=true&characterEncoding=utf-8, version : 1.2.4, lastPacketReceivedIdleMillis : 406943
2024-08-26 21:12:23.077 [http-nio-0.0.0.0-2024-exec-11] INFO datart.server.service.impl.SourceServiceImpl : jdbc source schema job has been created 5ad4570508f544b8930156b8ef95339d - Doris - interval 60
2024-08-26 21:12:24.896 [pool-2-thread-2] ERROR datart.server.service.impl.SourceServiceImpl : source schema sync error
java.lang.NullPointerException: null
at datart.data.provider.jdbc.DataTypeUtils.jdbcType2DataType(DataTypeUtils.java:61)
at datart.data.provider.jdbc.adapters.JdbcDataProviderAdapter.readTableColumn(JdbcDataProviderAdapter.java:193)
at datart.data.provider.jdbc.adapters.JdbcDataProviderAdapter.readTableColumn(JdbcDataProviderAdapter.java:176)
at datart.data.provider.JdbcDataProvider.readTableColumns(JdbcDataProvider.java:85)
at datart.data.provider.ProviderManager.readTableColumns(ProviderManager.java:114)
at datart.server.service.impl.DataProviderServiceImpl.readTableColumns(DataProviderServiceImpl.java:140)
at datart.server.job.SchemaSyncJob.execute(SchemaSyncJob.java:94)
at datart.server.service.impl.SourceServiceImpl.lambda$updateJdbcSourceSyncJob$2(SourceServiceImpl.java:464)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2024-08-26 21:12:24.976 [DatartScheduleCluster_Worker-4] ERROR datart.server.job.SchemaSyncJob : source schema sync error
java.lang.NullPointerException: null
at datart.data.provider.jdbc.DataTypeUtils.jdbcType2DataType(DataTypeUtils.java:61)
at datart.data.provider.jdbc.adapters.JdbcDataProviderAdapter.readTableColumn(JdbcDataProviderAdapter.java:193)
at datart.data.provider.jdbc.adapters.JdbcDataProviderAdapter.readTableColumn(JdbcDataProviderAdapter.java:176)
at datart.data.provider.JdbcDataProvider.readTableColumns(JdbcDataProvider.java:85)
at datart.data.provider.ProviderManager.readTableColumns(ProviderManager.java:114)
at datart.server.service.impl.DataProviderServiceImpl.readTableColumns(DataProviderServiceImpl.java:140)
at datart.server.job.SchemaSyncJob.execute(SchemaSyncJob.java:94)
at datart.server.job.SchemaSyncJob.execute(SchemaSyncJob.java:72)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)

Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
2024-08-26 20:55:52.592 [DatartScheduleCluster_Worker-3] WARN com.alibaba.druid.pool.DruidAbstractDataSource : discard long time none received connection. , jdbcUrl : jdbc:mysql://localhost:3333/datart?&allowMultiQueries=true&characterEncoding=utf-8, version : 1.2.4, lastPacketReceivedIdleMillis : 272659
2024-08-26 20:55:52.604 [DatartScheduleCluster_Worker-3] INFO d.data.provider.jdbc.DataSourceFactoryDruidImpl : druid data source created (DataSource-1830622635)
2024-08-26 20:55:52.608 [DatartScheduleCluster_Worker-3] INFO com.alibaba.druid.pool.DruidDataSource : {dataSource-5} inited
2024-08-26 20:55:53.286 [DatartScheduleCluster_Worker-3] ERROR datart.server.job.SchemaSyncJob : source schema sync error
java.lang.NullPointerException: null
at datart.data.provider.jdbc.DataTypeUtils.jdbcType2DataType(DataTypeUtils.java:61)
at datart.data.provider.jdbc.adapters.JdbcDataProviderAdapter.readTableColumn(JdbcDataProviderAdapter.java:193)
at datart.data.provider.jdbc.adapters.JdbcDataProviderAdapter.readTableColumn(JdbcDataProviderAdapter.java:176)
at datart.data.provider.JdbcDataProvider.readTableColumns(JdbcDataProvider.java:85)
at datart.data.provider.ProviderManager.readTableColumns(ProviderManager.java:114)
at datart.server.service.impl.DataProviderServiceImpl.readTableColumns(DataProviderServiceImpl.java:140)
at datart.server.job.SchemaSyncJob.execute(SchemaSyncJob.java:94)
at datart.server.job.SchemaSyncJob.execute(SchemaSyncJob.java:72)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)

@fohjseiou
Copy link

我这边也是的,请问解决了吗

@link2fun
Copy link

link2fun commented Sep 2, 2024

改下源码就是了

public static ValueType jdbcType2DataType(int jdbcType) {
        SqlTypeName sqlTypeName = SqlTypeName.getNameForJdbcType(jdbcType);
        SqlTypeFamily family;
        if (sqlTypeName == null) {
            family = SqlTypeFamily.getFamilyForJdbcType(jdbcType);
        } else {
            family = sqlTypeName.getFamily();
        }
// add start ======================
        if (family == null) {
            return ValueType.STRING;
        }
// add end ======================
        switch (family) {
            case NUMERIC:
                return ValueType.NUMERIC;
            case DATE:
            case TIME:
            case TIMESTAMP:
            case DATETIME:
                return ValueType.DATE;
            default:
                return ValueType.STRING;
        }
    }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants