Skip to content

[SPARK-31163][SQL] TruncateTableCommand with acl/permission should handle non-existent path #27923

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Ngone51 wants to merge 1 commit into from
Closed

[SPARK-31163][SQL] TruncateTableCommand with acl/permission should handle non-existent path #27923

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -508,8 +508,8 @@ case class TruncateTableCommand(
var optPermission: Option[FsPermission] = None
var optAcls: Option[java.util.List[AclEntry]] = None
if (!ignorePermissionAcl) {
val fileStatus = fs.getFileStatus(path)
try {
val fileStatus = fs.getFileStatus(path)
optPermission = Some(fileStatus.getPermission())
} catch {
case NonFatal(_) => // do nothing
Expand Down
21 changes: 21 additions & 0 deletions sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2084,6 +2084,27 @@ abstract class DDLSuite extends QueryTest with SQLTestUtils {
}
}

test("SPARK-31163: acl/permission should handle non-existed path when truncating table") {
withSQLConf(SQLConf.TRUNCATE_TABLE_IGNORE_PERMISSION_ACL.key -> "false") {
withTable("tab1") {
sql("CREATE TABLE tab1 (col1 STRING, col2 INT) USING parquet PARTITIONED BY (col2)")
sql("INSERT INTO tab1 SELECT 'one', 1")
checkAnswer(spark.table("tab1"), Row("one", 1))
val part = spark.sessionState.catalog.listPartitions(TableIdentifier("tab1")).head
val path = new File(part.location.getPath)
sql("TRUNCATE TABLE tab1")
// simulate incomplete/unsuccessful truncate
assert(path.exists())
path.delete()
assert(!path.exists())
// execute without java.io.FileNotFoundException
sql("TRUNCATE TABLE tab1")
// partition path should be re-created
Copy link
Contributor

@cloud-fan cloud-fan Mar 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this only affects partition path, or table path as well?

Copy link
Member Author

@Ngone51 Ngone51 Mar 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both, but can be more likely to happen on partition path since there's much more than table path.

assert(path.exists())
}
}
}

test("create temporary view with mismatched schema") {
withTable("tab1") {
spark.range(10).write.saveAsTable("tab1")
Expand Down